-- Logs begin at Mon 2022-09-05 16:20:26 UTC, end at Mon 2022-09-05 17:53:00 UTC. -- Sep 05 16:47:22.228840 dv-cinder-855964-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 05 16:47:22.239830 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 5 16:47:22 2022] *** Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: nodename: dv-cinder-855964-1-iscsi Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: machine: x86_64 Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: clock source: unix Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: pcre jit disabled Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: detected number of CPU cores: 4 Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: current working directory: / Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: detected binary path: /usr/bin/uwsgi-core Sep 05 16:47:22.242337 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: your processes number limit is 47680 Sep 05 16:47:22.242927 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: your memory page size is 4096 bytes Sep 05 16:47:22.242927 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: detected max file descriptor number: 2048 Sep 05 16:47:22.242927 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: lock engine: pthread robust mutexes Sep 05 16:47:22.242927 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: thunder lock: enabled Sep 05 16:47:22.242927 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 05 16:47:22.242927 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 05 16:47:22.281172 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: Python main interpreter initialized at 0x55b1073009c0 Sep 05 16:47:22.281172 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: python threads support enabled Sep 05 16:47:22.281172 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: your server socket listen backlog is limited to 100 connections Sep 05 16:47:22.281172 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: your mercy for graceful operations on workers is 90 seconds Sep 05 16:47:22.281563 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: mapped 403077 bytes (393 KB) for 2 cores Sep 05 16:47:22.281563 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: *** Operational MODE: preforking *** Sep 05 16:47:22.281853 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: *** uWSGI is running in multiple interpreter mode *** Sep 05 16:47:22.281853 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: spawned uWSGI master process (pid: 81509) Sep 05 16:47:22.282069 dv-cinder-855964-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 05 16:47:22.283114 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: spawned uWSGI worker 1 (pid: 81521, cores: 1) Sep 05 16:47:22.283114 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: spawned uWSGI worker 2 (pid: 81522, cores: 1) Sep 05 16:47:22.283114 dv-cinder-855964-1-iscsi devstack@keystone.service[81509]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 05 16:47:26.549231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 16:47:26.549231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: from cryptography.utils import int_from_bytes Sep 05 16:47:26.550063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 16:47:26.550063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: from cryptography.utils import int_from_bytes Sep 05 16:47:26.550179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 16:47:26.550179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: from cryptography.utils import int_from_bytes Sep 05 16:47:26.550583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 05 16:47:26.550583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: from cryptography.utils import int_from_bytes Sep 05 16:47:26.599405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 16:47:26.599623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 16:47:26.599705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 05 16:47:26.599777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] command line args: [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 16:47:26.599841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 05 16:47:26.599972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] command line args: [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 05 16:47:26.600061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 16:47:26.600196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ================================================================================ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 16:47:26.600347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 05 16:47:26.600432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ================================================================================ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 05 16:47:26.600499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] admin_token = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.600674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_dir = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.600766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] admin_token = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.600862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.600949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_dir = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_source = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] debug = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_source = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] debug = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.601919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_config_append = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.602932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_config_append = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_dir = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_dir = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.603846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.604966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.605161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.605677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.605677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.605840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.605903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.605960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606676 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.606923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.607128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.607250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.607619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.607754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.607754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] publish_errors = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.607919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] publish_errors = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.607999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.608900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] standard_threads = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] standard_threads = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.609927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_journal = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_journal = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_json = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_json = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_stderr = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_stderr = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_syslog = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_syslog = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 05 16:47:26.610847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.610930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.611911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612062 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.612904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.613879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.614828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.615992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.616988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.617991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.618950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.619903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.620932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.621931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622135 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.622889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.623927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.624924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.625951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.626981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.627941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.628920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.629958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.630109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.630198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.630341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.630424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.630638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.630728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.630911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.631964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.632939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.633932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.634881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.635842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.external = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.external = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.636815 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.password = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.password = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.token = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.token = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.637953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638749 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.638990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.639951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.640975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.641938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.642983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.643961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.644951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.645933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.646091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.646244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.646390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.646540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.646620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.646782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.646941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.647995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.648978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.649998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.650985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.651947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.652905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.653853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.654946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.655996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.656944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.657094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.657215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.657329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.657450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.657684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.657764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.657937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658727 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.658968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659438 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.659916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.660910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.661870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.662901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.663863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.664839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.665954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.666986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.667933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.668924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.669885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.670968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.671956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.672953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.673999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.674168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.674313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.674554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.674633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.674789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.674867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.675857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.676855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.677004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.677082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.677234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.677313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.677570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.677650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.677924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.678992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.679934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.680942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.681907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682062 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.682879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.683871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.684862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.685960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.686985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.687999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 16:47:26.688072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 05 16:47:26.688249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 05 16:47:26.717451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.717675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.717765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.717937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.718078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.718161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.726474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.726599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.754557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.754789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.754913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.755129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.808007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.808224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.856657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.856858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.857266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:26.857508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.013445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.013625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.013751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.013846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.014844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.037621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.037786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.037881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.038930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.039014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.039135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.039225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.050146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.050275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.062984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.063235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.063367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.063460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.063526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.063712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.063840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.064054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.064175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.064302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.064427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.064507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.064638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.064638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 05 16:47:27.080990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.081184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 16:47:27.081415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.081515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.081599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 16:47:27.081691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 16:47:27.081970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.082095 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.082174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 05 16:47:27.082249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.082430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.082563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.082671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.082767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.082954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.083183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.092495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.092778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.092902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.093094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.093180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.093277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.093353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.093590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.100505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.100630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.103392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.103518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.103684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.103815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.103975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.104104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.104267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.104363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.104557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.104727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.104813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.104955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.105038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.105167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.105657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.105792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.105913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.106006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.106205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.106336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.106477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.106603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.129036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.129166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.130494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.130690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.130866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.131014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.135561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.135687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.135842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.135970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.136105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.136223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.142417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.142546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.142703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.142831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.148719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.148849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.149008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.149134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.149299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.149447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.149561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.149655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.149869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.149995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.150165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.150312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.150439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.150566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.150690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.150853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.166244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.166543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.166824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.167104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.170404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.170975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.173630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.173930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.174208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.174510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.178164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.178755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.181353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.182030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.184583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.184898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.185200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.185327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.185481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.185612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.185752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.185882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.186009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.186133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.186337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.186609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.201913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.202209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.202486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.202756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.203275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.203571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.203843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.204114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.209686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.211008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.211257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.212572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.212790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.213072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.214090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.214382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.215779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.216070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.216387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.216677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.217078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.217286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.217427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.217675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.217961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.218365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.228209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.228525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.228811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.229103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.229501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.229788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.230019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.230151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.230311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.230916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.231200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.231581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.231847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.232127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.245728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.247410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.248630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.250339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.251529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.252995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.253238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.254479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.254760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.254889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.255171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.256506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.256804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.257216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.261717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.263178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.263588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.265012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.266091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.266497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.266790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.267077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.267871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.268301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.268591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.268865 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.275099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.275405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.275696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.276807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.277101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.277383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.282549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.282842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.283129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.283412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.283712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.283996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.284191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 05 16:47:27.284310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.284498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.284626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.284768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.284857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.285961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.286140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.286227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.286406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.286685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.286997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.287343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.287656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.287939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 05 16:47:27.321321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b1073009c0 pid: 81521 (default app) Sep 05 16:47:27.322261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b1073009c0 pid: 81522 (default app) Sep 05 16:47:27.324765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0cc9-699a-4f3f-8b71-500e845860ba None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:27.325059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0cc9-699a-4f3f-8b71-500e845860ba None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:27.325340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0cc9-699a-4f3f-8b71-500e845860ba None None] PATH_INFO: `/v3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:27.329628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Mon Sep 5 16:47:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:34.017255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a18e0-9784-4092-873b-ea91175249d7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:34.018110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a18e0-9784-4092-873b-ea91175249d7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:34.018243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a18e0-9784-4092-873b-ea91175249d7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:34.023836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:34.029097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:34.029568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:34.029707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:34.101487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: INFO dbcounter [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] Registered counter for database keystone Sep 05 16:47:34.123424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG oslo_db.sqlalchemy.engines [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81521) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 16:47:34.131259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writer thread running {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 16:47:34.326763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG passlib.handlers.bcrypt [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81521) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 16:47:34.327321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG passlib.handlers.bcrypt [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81521) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 16:47:34.391465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:34.393112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:34.532770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 05 16:47:35.002959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a18e0-9784-4092-873b-ea91175249d7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:35.081679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: INFO dbcounter [None req-904a18e0-9784-4092-873b-ea91175249d7 None None] Registered counter for database keystone Sep 05 16:47:35.108538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG oslo_db.sqlalchemy.engines [None req-904a18e0-9784-4092-873b-ea91175249d7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81522) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 05 16:47:35.118519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writer thread running {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 05 16:47:35.241619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a18e0-9784-4092-873b-ea91175249d7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:35.247131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f738a0-13d4-4acb-920f-655977b3d64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:35.247757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f738a0-13d4-4acb-920f-655977b3d64e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:35.248032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f738a0-13d4-4acb-920f-655977b3d64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:35.248326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f738a0-13d4-4acb-920f-655977b3d64e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:35.257605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f738a0-13d4-4acb-920f-655977b3d64e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:35.308868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING keystone.common.rbac_enforcer.enforcer [None req-95f738a0-13d4-4acb-920f-655977b3d64e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 16:47:35.309209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f738a0-13d4-4acb-920f-655977b3d64e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:35.319474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Mon Sep 5 16:47:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 16:47:35.328669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:35.344196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef67b-d9ad-4839-8d2e-1db27013ec96 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:35.350017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedc20d-4d6f-4f85-ad93-5c162c5ffc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:35.351157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedc20d-4d6f-4f85-ad93-5c162c5ffc65 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:35.351404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedc20d-4d6f-4f85-ad93-5c162c5ffc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:35.351874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedc20d-4d6f-4f85-ad93-5c162c5ffc65 admin admin] PATH_INFO: `/v3/projects/831cdea88c914362aff075e5d457eeb4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:35.353531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedc20d-4d6f-4f85-ad93-5c162c5ffc65 admin admin] RBAC: Authorizing `identity:get_project(project_id=831cdea88c914362aff075e5d457eeb4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:35.406243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ffedc20d-4d6f-4f85-ad93-5c162c5ffc65 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 05 16:47:35.406589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedc20d-4d6f-4f85-ad93-5c162c5ffc65 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:35.411406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Mon Sep 5 16:47:35 2022] GET /identity/v3/projects/831cdea88c914362aff075e5d457eeb4 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:37.297226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaf258-13aa-4065-ac70-d413827f9d6e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:37.298152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaf258-13aa-4065-ac70-d413827f9d6e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:37.298152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaf258-13aa-4065-ac70-d413827f9d6e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:37.304189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:37.311003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301360c1-76ec-4610-9e3e-0b2c9afde64b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:37.311423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301360c1-76ec-4610-9e3e-0b2c9afde64b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:37.311574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301360c1-76ec-4610-9e3e-0b2c9afde64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:37.344538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-301360c1-76ec-4610-9e3e-0b2c9afde64b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:37.344538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-301360c1-76ec-4610-9e3e-0b2c9afde64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:37.352779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 05 16:47:37.620978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faaf258-13aa-4065-ac70-d413827f9d6e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:37.639526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faaf258-13aa-4065-ac70-d413827f9d6e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:37.644980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e30b84-4123-41c1-9ad3-7bc6fa2af6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:37.645587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e30b84-4123-41c1-9ad3-7bc6fa2af6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:37.645876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e30b84-4123-41c1-9ad3-7bc6fa2af6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:37.646178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e30b84-4123-41c1-9ad3-7bc6fa2af6b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:37.655118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e30b84-4123-41c1-9ad3-7bc6fa2af6b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:37.656947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e30b84-4123-41c1-9ad3-7bc6fa2af6b2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:37.666122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Mon Sep 5 16:47:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 16:47:37.674407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301360c1-76ec-4610-9e3e-0b2c9afde64b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:37.683705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301360c1-76ec-4610-9e3e-0b2c9afde64b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:37.688862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd11dc-4a24-460c-8e97-57346dfc52ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:37.689496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd11dc-4a24-460c-8e97-57346dfc52ca admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:37.689778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd11dc-4a24-460c-8e97-57346dfc52ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:37.690050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd11dc-4a24-460c-8e97-57346dfc52ca admin admin] PATH_INFO: `/v3/users/fce1040c54ac43928ab07aa4e6b6d613` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:37.691497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd11dc-4a24-460c-8e97-57346dfc52ca admin admin] RBAC: Authorizing `identity:get_user(user_id=fce1040c54ac43928ab07aa4e6b6d613)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:37.693495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd11dc-4a24-460c-8e97-57346dfc52ca admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:37.695031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Mon Sep 5 16:47:37 2022] GET /identity/v3/users/fce1040c54ac43928ab07aa4e6b6d613 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:39.633787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90e85f-87cc-4fa4-bfdc-6dbce74d61e4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:39.634287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90e85f-87cc-4fa4-bfdc-6dbce74d61e4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:39.634287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90e85f-87cc-4fa4-bfdc-6dbce74d61e4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:39.635783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:39.641303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7388e26-1380-4de6-8eb5-66495edc22b8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:39.641699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7388e26-1380-4de6-8eb5-66495edc22b8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:39.641849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7388e26-1380-4de6-8eb5-66495edc22b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:39.661701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a7388e26-1380-4de6-8eb5-66495edc22b8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:39.662897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a7388e26-1380-4de6-8eb5-66495edc22b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:39.712064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6841e04-571b-48da-833e-262c7ea756e2 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:39.712392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6841e04-571b-48da-833e-262c7ea756e2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:39.712697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6841e04-571b-48da-833e-262c7ea756e2 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:39.714037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:47:39.719030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:39.719320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:39.719601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:39.748926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:39.762629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG passlib.handlers.bcrypt [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81522) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 05 16:47:39.763014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG passlib.handlers.bcrypt [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81522) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 05 16:47:39.811553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:39.813149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:39.914991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:47:40.009999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7388e26-1380-4de6-8eb5-66495edc22b8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.036698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:40.115904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7388e26-1380-4de6-8eb5-66495edc22b8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.150695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.151331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.151607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.151878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.163400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:47:40.172278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:40.173179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:47:40.174004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:40.179308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.182445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.267129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:40.280930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc68fe-0c5b-4f46-9b6a-1ff061d2af05 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.283381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:40.287150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0adc9e-e97b-4d77-b2f8-9ddde2f4b220 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.321404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.322288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.322573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.322849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.323692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:40.324005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:40.325532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:40.325646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.326219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.326506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.326817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.337905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:40.343266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.399089 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:40.400645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:40.444214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f352387-7973-4de0-b250-dc4f5fe48672 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.476398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.477257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.477533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.477803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] PATH_INFO: `/v3/users/fce1040c54ac43928ab07aa4e6b6d613` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.479312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] RBAC: Authorizing `identity:get_user(user_id=fce1040c54ac43928ab07aa4e6b6d613)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:40.480121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:40.481105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:40.482805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 9/16] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/users/fce1040c54ac43928ab07aa4e6b6d613 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:40.488861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.499461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 8/17] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 16:47:40.505126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.522928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:40.588936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201aa255-52c1-46b7-a204-88d8324d6035 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.605268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26c8f7-fdd7-4973-9fee-d1b9151972cd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.624321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.624961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.625244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.625522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.626660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:40.628112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:40.629706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:40.635685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.642089 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.642733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.643021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.643295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.653448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:40.654929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:47:40.670805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:40.671734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:47:40.672691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:47:40.672691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:47:40.672691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:47:40.672691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:40.677916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:40.713090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:40.739792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a259bc-23ef-4f84-a81b-6e5acaabdafc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.741364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:40.774433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be57a1-3078-4c98-bab8-1789cb4b906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.775042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be57a1-3078-4c98-bab8-1789cb4b906f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.775321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be57a1-3078-4c98-bab8-1789cb4b906f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.775649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be57a1-3078-4c98-bab8-1789cb4b906f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.776415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be57a1-3078-4c98-bab8-1789cb4b906f None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.domain.id=default, user.id=fce1040c54ac43928ab07aa4e6b6d613) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:40.776725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be57a1-3078-4c98-bab8-1789cb4b906f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:40.778178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be57a1-3078-4c98-bab8-1789cb4b906f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:40.779062 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a81ec8d-75e5-4a9f-8a3b-e449c88e6678 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:40.786170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/role_assignments?user.id=fce1040c54ac43928ab07aa4e6b6d613&scope.domain.id=default&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:40.814521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1ccda-49f3-4679-8c7f-2a97f3ef949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:40.815153 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1ccda-49f3-4679-8c7f-2a97f3ef949f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:40.815433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1ccda-49f3-4679-8c7f-2a97f3ef949f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:40.815707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1ccda-49f3-4679-8c7f-2a97f3ef949f None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:40.816400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1ccda-49f3-4679-8c7f-2a97f3ef949f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:40.816704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1ccda-49f3-4679-8c7f-2a97f3ef949f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:40.818309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1ccda-49f3-4679-8c7f-2a97f3ef949f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:40.838403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:42.783213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d81592-cd25-4d8e-8349-c8541b48cf37 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:42.784088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d81592-cd25-4d8e-8349-c8541b48cf37 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:42.784547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d81592-cd25-4d8e-8349-c8541b48cf37 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:42.786121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:42.801837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecbf57-cfe5-4bc4-a0c4-8be2294ff8f6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:42.803209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecbf57-cfe5-4bc4-a0c4-8be2294ff8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:42.803712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecbf57-cfe5-4bc4-a0c4-8be2294ff8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:42.833148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6aecbf57-cfe5-4bc4-a0c4-8be2294ff8f6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:42.834818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6aecbf57-cfe5-4bc4-a0c4-8be2294ff8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:42.918528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:43.117014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3281d-6773-4395-b89b-466e03b487ec None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.117739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3281d-6773-4395-b89b-466e03b487ec None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.118211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3281d-6773-4395-b89b-466e03b487ec None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:43.119796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:43.125128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9716d-60df-41fd-a966-ceb87eb3d623 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.125624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9716d-60df-41fd-a966-ceb87eb3d623 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.126100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9716d-60df-41fd-a966-ceb87eb3d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:43.157108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3ab9716d-60df-41fd-a966-ceb87eb3d623 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:43.158611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3ab9716d-60df-41fd-a966-ceb87eb3d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:43.204540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:43.273245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 12/25] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:43.399086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3281d-6773-4395-b89b-466e03b487ec None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:43.534506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3281d-6773-4395-b89b-466e03b487ec None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:43.581620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429045cf-db51-4116-a636-8689b5b6c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:43.582643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429045cf-db51-4116-a636-8689b5b6c978 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.583122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429045cf-db51-4116-a636-8689b5b6c978 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.583607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429045cf-db51-4116-a636-8689b5b6c978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:43.595601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429045cf-db51-4116-a636-8689b5b6c978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-429045cf-db51-4116-a636-8689b5b6c978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:47:43.604742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:43.605905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:43.606626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:47:43.608925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 14/26] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:47:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:43.617292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9716d-60df-41fd-a966-ceb87eb3d623 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:43.658653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace513b8-bfae-4d5c-b4ce-4eea3273ec24 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.659262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace513b8-bfae-4d5c-b4ce-4eea3273ec24 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.659726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace513b8-bfae-4d5c-b4ce-4eea3273ec24 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:43.661688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 15/27] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:43.666892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bcfa5-1c48-4156-8dc3-ee59c3e5298d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.667400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bcfa5-1c48-4156-8dc3-ee59c3e5298d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.667877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bcfa5-1c48-4156-8dc3-ee59c3e5298d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:43.698433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-714bcfa5-1c48-4156-8dc3-ee59c3e5298d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:43.699958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-714bcfa5-1c48-4156-8dc3-ee59c3e5298d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:43.731557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:43.802864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 16/28] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:43.805906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bcfa5-1c48-4156-8dc3-ee59c3e5298d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:43.846152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9716d-60df-41fd-a966-ceb87eb3d623 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:43.909762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bcfa5-1c48-4156-8dc3-ee59c3e5298d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:43.934844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68dc228-3356-47f8-b472-42d231168152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:43.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc228-3356-47f8-b472-42d231168152 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.936398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc228-3356-47f8-b472-42d231168152 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.936897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68dc228-3356-47f8-b472-42d231168152 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:43.937768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dc228-3356-47f8-b472-42d231168152 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:43.938256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dc228-3356-47f8-b472-42d231168152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:43.939857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68dc228-3356-47f8-b472-42d231168152 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:43.969189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 13/29] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:43.976719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0182740-dbb4-496a-ac6a-22ece6b1bf93 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.977304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0182740-dbb4-496a-ac6a-22ece6b1bf93 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.978133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0182740-dbb4-496a-ac6a-22ece6b1bf93 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:43.979830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 14/30] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:47:43.981134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0182740-dbb4-496a-ac6a-22ece6b1bf93 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:43.984673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff953831-6097-44e4-b866-17a207168f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:43.985327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff953831-6097-44e4-b866-17a207168f74 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:43.985611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff953831-6097-44e4-b866-17a207168f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:43.985882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff953831-6097-44e4-b866-17a207168f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.005681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff953831-6097-44e4-b866-17a207168f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:44.007181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff953831-6097-44e4-b866-17a207168f74 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ff953831-6097-44e4-b866-17a207168f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:47:44.024022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:44.024922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:47:44.026134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:47:44.026134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:47:44.026134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:47:44.026134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 17/31] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:44.026335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:44.026401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f8ff2-41d5-48b6-b081-309f27ae173f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.027397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f8ff2-41d5-48b6-b081-309f27ae173f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.027397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f8ff2-41d5-48b6-b081-309f27ae173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.046109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e48f8ff2-41d5-48b6-b081-309f27ae173f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:44.047310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e48f8ff2-41d5-48b6-b081-309f27ae173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:44.049365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:44.156291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 18/32] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:44.156956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f8ff2-41d5-48b6-b081-309f27ae173f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:44.223898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0182740-dbb4-496a-ac6a-22ece6b1bf93 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:44.283536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:44.284194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.284475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.284749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] PATH_INFO: `/v3/users/fce1040c54ac43928ab07aa4e6b6d613` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.286556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] RBAC: Authorizing `identity:get_user(user_id=fce1040c54ac43928ab07aa4e6b6d613)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:44.289435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:44.291836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 15/33] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:47:43 2022] GET /identity/v3/users/fce1040c54ac43928ab07aa4e6b6d613 => generated 247 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:44.292602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:44.358589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f8ff2-41d5-48b6-b081-309f27ae173f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:44.424639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:44.436760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45267001-84bb-4b07-bbad-d2d6d8f448d3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:44.446057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:44.446915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.447392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.447850 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.448837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:44.449363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:44.450927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:44.462926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:44.470604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 19/34] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:44.471405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:44.479970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc05be-879b-4bab-9fc8-d2402cf52343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:44.481281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc05be-879b-4bab-9fc8-d2402cf52343 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.481568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc05be-879b-4bab-9fc8-d2402cf52343 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.481841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc05be-879b-4bab-9fc8-d2402cf52343 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.482524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc05be-879b-4bab-9fc8-d2402cf52343 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:44.484590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc05be-879b-4bab-9fc8-d2402cf52343 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:44.571969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e4ded-9149-481c-a7a2-36ad6823cea5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:44.606039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:44.606666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.606937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.607206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.608367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:44.609808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:44.616357 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 20/35] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:44.616357 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:44.635537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 16/36] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Sep 5 16:47:44 2022] POST /identity/v3/roles => generated 225 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:47:44.637057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dcbf3-d97b-4aef-9a16-d26479cea9be None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.637344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dcbf3-d97b-4aef-9a16-d26479cea9be None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.637624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753dcbf3-d97b-4aef-9a16-d26479cea9be None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.639051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 17/37] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:44.639544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753dcbf3-d97b-4aef-9a16-d26479cea9be None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:44.694368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:44.739236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c909e3c-9c99-4ecd-ac4a-d9555dc2b512 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:44.765125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753dcbf3-d97b-4aef-9a16-d26479cea9be None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:44.779953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab99cdb-14d0-4371-8331-19fa3a4b6323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:44.780702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99cdb-14d0-4371-8331-19fa3a4b6323 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.781911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99cdb-14d0-4371-8331-19fa3a4b6323 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.781911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab99cdb-14d0-4371-8331-19fa3a4b6323 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.781911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab99cdb-14d0-4371-8331-19fa3a4b6323 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:44.783261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab99cdb-14d0-4371-8331-19fa3a4b6323 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:44.808002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:44.808002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.808002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.808362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.808980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:44.812171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:44.829766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:44.843713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 18/38] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:47:44 2022] POST /identity/v3/projects => generated 301 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:47:44.844679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:44.912217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 21/39] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Sep 5 16:47:44 2022] POST /identity/v3/roles => generated 219 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:47:44.912217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7336e-098d-48c7-9b35-bf29c087b87c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.912217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7336e-098d-48c7-9b35-bf29c087b87c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.912217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7336e-098d-48c7-9b35-bf29c087b87c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.912217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 22/40] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:44.913593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c34a-0eb0-48e2-8f95-e3d7c9776c47 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:44.913891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c34a-0eb0-48e2-8f95-e3d7c9776c47 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:44.914184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c34a-0eb0-48e2-8f95-e3d7c9776c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:44.947180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c505c34a-0eb0-48e2-8f95-e3d7c9776c47 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:44.948955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c505c34a-0eb0-48e2-8f95-e3d7c9776c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:44.979623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291995-3354-44f4-b761-a175a2cb7411 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:45.047682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 23/41] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:45.048989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b657ff-e554-4cfe-b147-78d845434389 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:45.049278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b657ff-e554-4cfe-b147-78d845434389 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:45.049551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b657ff-e554-4cfe-b147-78d845434389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:45.077719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-10b657ff-e554-4cfe-b147-78d845434389 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:45.079201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-10b657ff-e554-4cfe-b147-78d845434389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:45.144314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:45.145841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:45.146588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:45.146868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:45.147606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] PATH_INFO: `/v3/domains/default/users/fce1040c54ac43928ab07aa4e6b6d613/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:45.150159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fce1040c54ac43928ab07aa4e6b6d613, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:45.152316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:45.166420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 24/42] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:45.176890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 19/43] 10.222.0.23 () {58 vars in 1361 bytes} [Mon Sep 5 16:47:44 2022] PUT /identity/v3/domains/default/users/fce1040c54ac43928ab07aa4e6b6d613/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:47:45.329233 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b657ff-e554-4cfe-b147-78d845434389 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:45.431641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:45.433149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b657ff-e554-4cfe-b147-78d845434389 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:45.468942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:45.469620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:45.469903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:45.470178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:45.471274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:45.472890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:45.475203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 25/44] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:45.480130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:45.498810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:45.537188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:45.548046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8bc14-45b0-4417-a17f-c2dd5b270c97 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:45.586673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d33d5-367e-400b-8346-2dcc478fac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:45.587245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d33d5-367e-400b-8346-2dcc478fac9d None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:45.587503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d33d5-367e-400b-8346-2dcc478fac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:45.587783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d33d5-367e-400b-8346-2dcc478fac9d None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:45.588491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d33d5-367e-400b-8346-2dcc478fac9d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:45.589091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c2e3c-f93b-431d-9d80-d72ae5b76a99 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:45.589901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d33d5-367e-400b-8346-2dcc478fac9d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:45.606073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 20/45] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Sep 5 16:47:45 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:47:45.628452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ebceb-aa6e-4600-b530-325f015c85d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:45.629094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ebceb-aa6e-4600-b530-325f015c85d5 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:45.629375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ebceb-aa6e-4600-b530-325f015c85d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:45.629652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ebceb-aa6e-4600-b530-325f015c85d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:45.630437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ebceb-aa6e-4600-b530-325f015c85d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:45.631932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ebceb-aa6e-4600-b530-325f015c85d5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:45.672183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 26/46] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:47:45 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:47:47.073810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a153a-57ee-40cd-a404-651f75b0a9ba None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:47.074361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a153a-57ee-40cd-a404-651f75b0a9ba None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:47.074361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a153a-57ee-40cd-a404-651f75b0a9ba None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:47.075707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 21/47] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:47.081241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20505e-3631-438d-af4a-f4d2ff8d451d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:47.081665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20505e-3631-438d-af4a-f4d2ff8d451d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:47.081873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20505e-3631-438d-af4a-f4d2ff8d451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:47.205280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ea20505e-3631-438d-af4a-f4d2ff8d451d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:47.206494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea20505e-3631-438d-af4a-f4d2ff8d451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:47.277121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 27/48] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:47:47.543272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a153a-57ee-40cd-a404-651f75b0a9ba None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:47.566357 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0820d-c6e2-498b-a80e-fbb1d58bc65d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:47.566650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0820d-c6e2-498b-a80e-fbb1d58bc65d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:47.567370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0820d-c6e2-498b-a80e-fbb1d58bc65d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:47.568903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 28/49] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:47:47.573726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e7026-dc1f-437d-81f5-bc30b8babd7b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:47.574010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e7026-dc1f-437d-81f5-bc30b8babd7b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:47.574512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e7026-dc1f-437d-81f5-bc30b8babd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:47.592268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5c5e7026-dc1f-437d-81f5-bc30b8babd7b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:47.593466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5c5e7026-dc1f-437d-81f5-bc30b8babd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:47.624689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:47.652458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a153a-57ee-40cd-a404-651f75b0a9ba None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:47.664306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 29/50] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:47:47.669668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:47.688109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:47.688789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:47.689073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:47.689353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:47.699537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:47:47.704539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:47.705552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:47:47.706670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 22/51] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:47:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:47.711518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e7026-dc1f-437d-81f5-bc30b8babd7b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:47.810566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e7026-dc1f-437d-81f5-bc30b8babd7b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:47.845261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:47.845873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:47.846153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:47.846430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:47.847093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:47.847410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:47.848864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:47.857699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 30/52] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:47.864058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:47.925941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:47.956554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:47.968717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b7592-d918-453a-b529-2e07fdca46ef None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:48.002647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:48.003315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:48.003594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:48.003871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] PATH_INFO: `/v3/users/fce1040c54ac43928ab07aa4e6b6d613` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:48.005056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] RBAC: Authorizing `identity:get_user(user_id=fce1040c54ac43928ab07aa4e6b6d613)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:48.006585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:48.008244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 23/53] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:47:47 2022] GET /identity/v3/users/fce1040c54ac43928ab07aa4e6b6d613 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:48.014008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:48.031011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014ae4d-ca47-4e99-9648-98c2e2a0a79e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:48.061783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:48.065421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:48.066063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:48.066349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:48.066856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:48.067841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:48.069514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:48.071128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 31/54] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:48.076772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:48.115564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497602da-4d34-49e0-9589-903210a01d83 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:48.151222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:48.151860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:48.152168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:48.152459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:48.153592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:48.155080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:48.156701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 24/55] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:47:48.162778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:48.174548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:48.181447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983009c-2069-4ca2-bd98-3b10112eb875 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:48.216257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657a95d-3a19-45e9-a0ed-187196412ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:48.216844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657a95d-3a19-45e9-a0ed-187196412ef2 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:48.217124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657a95d-3a19-45e9-a0ed-187196412ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:48.217400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657a95d-3a19-45e9-a0ed-187196412ef2 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:48.218042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657a95d-3a19-45e9-a0ed-187196412ef2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:48.219478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657a95d-3a19-45e9-a0ed-187196412ef2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:48.247531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 32/56] 10.222.0.23 () {60 vars in 1140 bytes} [Mon Sep 5 16:47:48 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:47:48.263723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f71304-eadd-4bfb-a298-8f6c4a6dcc62 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:48.279945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:48.304736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd0a31-b9ac-40dc-a4bf-48331560c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:48.305380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd0a31-b9ac-40dc-a4bf-48331560c52a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:48.305668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd0a31-b9ac-40dc-a4bf-48331560c52a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:48.305948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd0a31-b9ac-40dc-a4bf-48331560c52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:48.306704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd0a31-b9ac-40dc-a4bf-48331560c52a None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.domain.id=default, user.id=fce1040c54ac43928ab07aa4e6b6d613) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:48.307001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd0a31-b9ac-40dc-a4bf-48331560c52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:48.308544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd0a31-b9ac-40dc-a4bf-48331560c52a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:48.316706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 25/57] 10.222.0.23 () {56 vars in 1322 bytes} [Mon Sep 5 16:47:48 2022] GET /identity/v3/role_assignments?user.id=fce1040c54ac43928ab07aa4e6b6d613&scope.domain.id=default&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:50.147623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748eb54-545e-4b28-aac7-b41ef566b801 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:50.148199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748eb54-545e-4b28-aac7-b41ef566b801 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:50.148199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748eb54-545e-4b28-aac7-b41ef566b801 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:50.149577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 33/58] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:50.155932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51e6c9-d5ce-4162-bd19-719e4f9423e8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:50.156422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51e6c9-d5ce-4162-bd19-719e4f9423e8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:50.156568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51e6c9-d5ce-4162-bd19-719e4f9423e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:50.178434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3d51e6c9-d5ce-4162-bd19-719e4f9423e8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:50.179991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3d51e6c9-d5ce-4162-bd19-719e4f9423e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:50.258517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 26/59] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:50.525352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748eb54-545e-4b28-aac7-b41ef566b801 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:50.635602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748eb54-545e-4b28-aac7-b41ef566b801 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:50.671140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded79e1-160f-4963-a048-7acc319aed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:50.671760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded79e1-160f-4963-a048-7acc319aed86 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:50.672036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded79e1-160f-4963-a048-7acc319aed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:50.672335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded79e1-160f-4963-a048-7acc319aed86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:50.673563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded79e1-160f-4963-a048-7acc319aed86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:50.675014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded79e1-160f-4963-a048-7acc319aed86 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:50.676614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 34/60] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:50.687049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51e6c9-d5ce-4162-bd19-719e4f9423e8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:50.795064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51e6c9-d5ce-4162-bd19-719e4f9423e8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:50.802339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:50.829842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0d08c-7ffc-4b4b-8d7f-5c89d7602a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:50.830618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0d08c-7ffc-4b4b-8d7f-5c89d7602a63 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:50.830944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0d08c-7ffc-4b4b-8d7f-5c89d7602a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:50.831266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0d08c-7ffc-4b4b-8d7f-5c89d7602a63 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:50.831913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0d08c-7ffc-4b4b-8d7f-5c89d7602a63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:50.833484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0d08c-7ffc-4b4b-8d7f-5c89d7602a63 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:50.860751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 27/61] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:47:50 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:47:53.924239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea73c1-b628-4034-afb1-3bf321cdac13 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:53.925267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea73c1-b628-4034-afb1-3bf321cdac13 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:53.925743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea73c1-b628-4034-afb1-3bf321cdac13 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:53.927434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 35/62] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:53.932883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff73a-fe4c-4479-802e-1a76f974354f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:53.933691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff73a-fe4c-4479-802e-1a76f974354f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:53.934164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddff73a-fe4c-4479-802e-1a76f974354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:53.957465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3ddff73a-fe4c-4479-802e-1a76f974354f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:53.959011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3ddff73a-fe4c-4479-802e-1a76f974354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:53.978344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f70c55-12b3-43f9-9763-bae83909a73f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:53.978928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f70c55-12b3-43f9-9763-bae83909a73f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:53.979389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f70c55-12b3-43f9-9763-bae83909a73f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:53.981260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 36/63] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:47:53.987460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030f427-126e-458b-9eb4-d2d772545376 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:53.988007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030f427-126e-458b-9eb4-d2d772545376 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:53.988928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d030f427-126e-458b-9eb4-d2d772545376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.010648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d030f427-126e-458b-9eb4-d2d772545376 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:54.012367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d030f427-126e-458b-9eb4-d2d772545376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:54.026966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:54.089827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 37/64] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:47:54.114408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 28/65] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:47:54.171105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce05cf-27f2-45c1-aaea-1057d142ba92 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.171729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce05cf-27f2-45c1-aaea-1057d142ba92 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.172208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce05cf-27f2-45c1-aaea-1057d142ba92 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.174009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 38/66] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:54.179136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb4655-3ffe-4c5f-a854-51e4a00ad686 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.179373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb4655-3ffe-4c5f-a854-51e4a00ad686 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.179661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb4655-3ffe-4c5f-a854-51e4a00ad686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.200659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-53fb4655-3ffe-4c5f-a854-51e4a00ad686 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:54.202178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-53fb4655-3ffe-4c5f-a854-51e4a00ad686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:54.272928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 29/67] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:54.408309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce05cf-27f2-45c1-aaea-1057d142ba92 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:54.427412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb4655-3ffe-4c5f-a854-51e4a00ad686 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:54.476612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:54.583431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce05cf-27f2-45c1-aaea-1057d142ba92 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:54.604639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb4655-3ffe-4c5f-a854-51e4a00ad686 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:54.619792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77737faf-c119-449b-9adc-e6b0faec17e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:54.620466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77737faf-c119-449b-9adc-e6b0faec17e9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.620911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77737faf-c119-449b-9adc-e6b0faec17e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.621191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77737faf-c119-449b-9adc-e6b0faec17e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.632261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77737faf-c119-449b-9adc-e6b0faec17e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-77737faf-c119-449b-9adc-e6b0faec17e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:47:54.637156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:54.637987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:47:54.638942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:47:54.643277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 39/68] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:54.644714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:54.644905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e53d0e-5c8f-4c6f-9398-dd1cdeca60fa None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.645189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e53d0e-5c8f-4c6f-9398-dd1cdeca60fa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.645353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.645529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e53d0e-5c8f-4c6f-9398-dd1cdeca60fa None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.645680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.646013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.646789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 40/69] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:54.647486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e53d0e-5c8f-4c6f-9398-dd1cdeca60fa None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:54.657151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:54.662462 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:47:54.667339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:54.668268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:47:54.669019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 30/70] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:47:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:54.669646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:54.750021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:54.750999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e53d0e-5c8f-4c6f-9398-dd1cdeca60fa None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:54.768763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869dbe3-0ff0-4887-a0de-7c1ca30e7311 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:54.786773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a750b-68d8-4dea-a2be-f8ebbc42fbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:54.787405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a750b-68d8-4dea-a2be-f8ebbc42fbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.787677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a750b-68d8-4dea-a2be-f8ebbc42fbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.787917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a750b-68d8-4dea-a2be-f8ebbc42fbd9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.798053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a750b-68d8-4dea-a2be-f8ebbc42fbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1e1a750b-68d8-4dea-a2be-f8ebbc42fbd9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:47:54.803070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:54.803913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:47:54.804798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 41/71] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:47:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:54.805453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681cd1-a196-4335-873a-caeccff6df2b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.805738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681cd1-a196-4335-873a-caeccff6df2b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.806011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03681cd1-a196-4335-873a-caeccff6df2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.807912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bbae7f-2a84-4068-ac36-97a71bec23d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:54.808571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbae7f-2a84-4068-ac36-97a71bec23d1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.808908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbae7f-2a84-4068-ac36-97a71bec23d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.809137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbae7f-2a84-4068-ac36-97a71bec23d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.809808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbae7f-2a84-4068-ac36-97a71bec23d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:54.810117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbae7f-2a84-4068-ac36-97a71bec23d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:54.811517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bbae7f-2a84-4068-ac36-97a71bec23d1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:54.820607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 31/72] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:47:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:54.821709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b428d0-e209-4cf3-8dc9-f3cf8887a3fb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.822000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b428d0-e209-4cf3-8dc9-f3cf8887a3fb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.822278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b428d0-e209-4cf3-8dc9-f3cf8887a3fb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.823510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 32/73] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:54.824284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b428d0-e209-4cf3-8dc9-f3cf8887a3fb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:54.829315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-03681cd1-a196-4335-873a-caeccff6df2b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:54.830489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03681cd1-a196-4335-873a-caeccff6df2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:54.902767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:54.913935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 42/74] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:54.914847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03681cd1-a196-4335-873a-caeccff6df2b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:54.949140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b428d0-e209-4cf3-8dc9-f3cf8887a3fb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:54.995114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:54.995840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:54.996229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:54.996581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:54.997652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:54.997652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:54.999132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.009123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 33/75] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:55.009977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.023679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.035706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03681cd1-a196-4335-873a-caeccff6df2b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.081788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7f49e-c3dd-452f-b2d5-1b8064ef22c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.082999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7f49e-c3dd-452f-b2d5-1b8064ef22c2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.083493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7f49e-c3dd-452f-b2d5-1b8064ef22c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.083940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7f49e-c3dd-452f-b2d5-1b8064ef22c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.084785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7f49e-c3dd-452f-b2d5-1b8064ef22c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:55.085231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7f49e-c3dd-452f-b2d5-1b8064ef22c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.086883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7f49e-c3dd-452f-b2d5-1b8064ef22c2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.095996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 43/76] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:47:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:55.097393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ec6a-b844-411a-9186-822d21f7ed46 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.097867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ec6a-b844-411a-9186-822d21f7ed46 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.098295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9ec6a-b844-411a-9186-822d21f7ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.120365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-08b9ec6a-b844-411a-9186-822d21f7ed46 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:55.121576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-08b9ec6a-b844-411a-9186-822d21f7ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:55.135085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc80ca-b230-4167-bbb9-25217723802f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.177643 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.178298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.178584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.178866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.182479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.194848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.196485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.198398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 34/77] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:55.199183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.204589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 44/78] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:55.205670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9ec6a-b844-411a-9186-822d21f7ed46 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.278784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.319313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cfffc-ef5d-4ce3-b6e6-72f8dcbda7e0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.328867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9ec6a-b844-411a-9186-822d21f7ed46 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.357922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.358611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.358917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.359202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] PATH_INFO: `/v3/users/fce1040c54ac43928ab07aa4e6b6d613` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.360391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] RBAC: Authorizing `identity:get_user(user_id=fce1040c54ac43928ab07aa4e6b6d613)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.361873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.363512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 35/79] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/users/fce1040c54ac43928ab07aa4e6b6d613 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:55.365835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.366687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.367144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.367648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.370019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.371589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.373381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.375179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 45/80] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:55.380718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.439553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.479126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be49ccd-8284-4ae5-80ab-56d7443bf8be None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.492042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1d9a3-2b7d-4075-9cce-b61170222024 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.509551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.515317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.515965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.516285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.516565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.527143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:47:55.527946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.528607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.528944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.529168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:47:55.531997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:47:55.532996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 36/81] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:47:55.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.607486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.609014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.642013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ff6e4-8a6d-4a39-987d-221b8d52913e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.667278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.675632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.676291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.676578 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.676881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.711091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 46/82] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:47:55.711938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.738864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.749020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.750489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.808664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587136e-b497-4dd0-9a8e-d27a38dee410 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.842219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.842826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.843106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.843377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.848259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 37/83] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:47:55.849246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:55.868914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.911502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.913109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.938973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:55.950747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e359b-cc6c-4d3c-b4dc-98d972b60ab1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:55.984938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:55.985603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:55.985881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:55.986145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:55.987256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:55.988763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:55.990518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 38/84] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:55.991288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.017583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 47/85] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:47:56.018447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.074858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:56.093240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba965563-2245-408c-9cbd-e950d2848add None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.117414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44410e3c-1f26-4c7c-a870-5fc7be5f82c9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.127916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.128580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.128864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.129135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.129798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:56.130604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.132023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.140378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 39/86] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:47:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.141006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.155800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.156487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.156766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.157044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.158218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.159682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.161570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 48/87] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.162305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.171210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:56.249243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5375-8680-4f8a-a79e-e4951d26f1c4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.262998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e6266d-ae67-49b3-9183-6edc0e0d01bc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.282926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.283577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.283858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.284136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.285362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.286891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.288576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 40/88] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.289345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.294888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:56.307114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.307831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.308120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.308419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.309067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.310526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.338075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 49/89] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:47:56 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:47:56.338862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.407014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:56.422234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfaf8b4-d3a8-4352-af7a-4eb0be104968 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.460814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cf5ab-b131-4935-a200-a39ca9a02e02 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.462628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.463292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.463569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.463841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.465041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.466512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.468230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 41/90] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.469113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.503193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.504102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.504452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.504735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.506360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.507882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.509699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 50/91] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.510310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.575294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:56.596200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939a56c-17d7-48c2-bdc2-0483acb45428 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.638918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.639546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.639832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.640114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.640898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=e46173bc1d274512ae45ef02806797bc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:56.641208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.642674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.647653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df0a0e-769a-4cc0-91b1-ca8b071cde35 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.648647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 42/92] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/role_assignments?user.id=e46173bc1d274512ae45ef02806797bc&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.649879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.686824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:56.694238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.695138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.695426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.695705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.696996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=fce1040c54ac43928ab07aa4e6b6d613) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:56.697329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.699114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.708341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 51/93] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/role_assignments?user.id=fce1040c54ac43928ab07aa4e6b6d613&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.709533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:56.795593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7140303-f0cf-4bbf-ade4-b65e1275e58c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.855995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.856566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.856852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.857123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:56.857865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] RBAC: Adding query filter params (role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=e46173bc1d274512ae45ef02806797bc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:56.858174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:56.865622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:56.874996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 43/94] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/role_assignments?user.id=e46173bc1d274512ae45ef02806797bc&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:56.899199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3604e1-2f7b-4a9a-8869-d5d19b8ae20a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:56.916582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:56.940435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:56.941054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:56.941335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:56.941612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:57.017203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:57.018685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:57.126529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 52/95] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:47:57.132645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:57.238615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:57.256485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6a333-a6e3-4822-9ea8-4b24aa8f95e9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:57.302195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4e9cf-9cbf-480d-b220-95a48c5fbe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:57.302905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4e9cf-9cbf-480d-b220-95a48c5fbe59 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:57.303191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4e9cf-9cbf-480d-b220-95a48c5fbe59 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:57.303467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4e9cf-9cbf-480d-b220-95a48c5fbe59 None admin] PATH_INFO: `/v3/projects/a66dafb589fd43558a1aa89dd6e16031` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:57.304632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4e9cf-9cbf-480d-b220-95a48c5fbe59 None admin] RBAC: Authorizing `identity:get_project(project_id=a66dafb589fd43558a1aa89dd6e16031)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:57.306108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4e9cf-9cbf-480d-b220-95a48c5fbe59 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:57.307766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 44/96] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:47:57 2022] GET /identity/v3/projects/a66dafb589fd43558a1aa89dd6e16031 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:57.314200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:57.340575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:57.421961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d3786-cdba-4d4d-838c-99662026b709 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:47:57.457199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d19919-88cb-4c74-8f44-1ae1ae9efa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:47:57.457820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d19919-88cb-4c74-8f44-1ae1ae9efa90 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:57.458260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d19919-88cb-4c74-8f44-1ae1ae9efa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:57.458793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d19919-88cb-4c74-8f44-1ae1ae9efa90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:57.459192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d19919-88cb-4c74-8f44-1ae1ae9efa90 None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=a66dafb589fd43558a1aa89dd6e16031, user.id=e46173bc1d274512ae45ef02806797bc) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:47:57.459499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d19919-88cb-4c74-8f44-1ae1ae9efa90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:47:57.460988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d19919-88cb-4c74-8f44-1ae1ae9efa90 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:47:57.466824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 53/97] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:47:57 2022] GET /identity/v3/role_assignments?user.id=e46173bc1d274512ae45ef02806797bc&scope.project.id=a66dafb589fd43558a1aa89dd6e16031&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:47:59.138679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ccb36-2b20-41b9-8398-7f9f09ca2c8e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.139233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ccb36-2b20-41b9-8398-7f9f09ca2c8e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.139233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ccb36-2b20-41b9-8398-7f9f09ca2c8e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.140883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 45/98] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:59.146372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11c1ea-22a6-4698-8fea-7db08e231b7d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.146884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11c1ea-22a6-4698-8fea-7db08e231b7d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.146884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11c1ea-22a6-4698-8fea-7db08e231b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.170169 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ca11c1ea-22a6-4698-8fea-7db08e231b7d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:59.171589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ca11c1ea-22a6-4698-8fea-7db08e231b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:59.233470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59ee49-a4ca-426b-a7e9-448f7b84c7d0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.233979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59ee49-a4ca-426b-a7e9-448f7b84c7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.234417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59ee49-a4ca-426b-a7e9-448f7b84c7d0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.235894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 46/99] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:59.250661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 54/100] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:59.252976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395d3a-ade8-437d-b408-abbedf60fbc1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.253418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395d3a-ade8-437d-b408-abbedf60fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.253825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395d3a-ade8-437d-b408-abbedf60fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.273400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-16395d3a-ade8-437d-b408-abbedf60fbc1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:59.274796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-16395d3a-ade8-437d-b408-abbedf60fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:59.344523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 47/101] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:59.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ed2ed-2baa-4317-8d62-dbca6de65043 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.498099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ed2ed-2baa-4317-8d62-dbca6de65043 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.498274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ed2ed-2baa-4317-8d62-dbca6de65043 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.499663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 55/102] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:59.509250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a7534-80d2-4c3f-909b-10fb63c24872 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.509546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a7534-80d2-4c3f-909b-10fb63c24872 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.509835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a7534-80d2-4c3f-909b-10fb63c24872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.537773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-432a7534-80d2-4c3f-909b-10fb63c24872 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:59.539011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-432a7534-80d2-4c3f-909b-10fb63c24872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:59.634562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 48/103] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:59.818447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b060d-5c83-4ed2-86de-178505d48d78 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.818629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931822b1-2ed5-4a49-b495-df222732277b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.818718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b060d-5c83-4ed2-86de-178505d48d78 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.818787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931822b1-2ed5-4a49-b495-df222732277b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.818853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b060d-5c83-4ed2-86de-178505d48d78 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.818916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931822b1-2ed5-4a49-b495-df222732277b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.818988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 56/104] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:47:59.819061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 49/105] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:47:59.823409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2071b8-aba3-4f74-a232-98cca8b3d08e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.823905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2071b8-aba3-4f74-a232-98cca8b3d08e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.824400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2071b8-aba3-4f74-a232-98cca8b3d08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.831334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32117f1-e755-4a17-b7b0-dd74e64c7480 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:47:59.831677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32117f1-e755-4a17-b7b0-dd74e64c7480 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:47:59.831961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32117f1-e755-4a17-b7b0-dd74e64c7480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:47:59.839108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:59.847348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1f2071b8-aba3-4f74-a232-98cca8b3d08e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:59.849897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1f2071b8-aba3-4f74-a232-98cca8b3d08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:59.860667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d32117f1-e755-4a17-b7b0-dd74e64c7480 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:47:59.862134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d32117f1-e755-4a17-b7b0-dd74e64c7480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:47:59.872640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:47:59.938299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 57/106] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:59.939776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2071b8-aba3-4f74-a232-98cca8b3d08e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:47:59.941318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 50/107] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:47:59.942168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32117f1-e755-4a17-b7b0-dd74e64c7480 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.129961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2071b8-aba3-4f74-a232-98cca8b3d08e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.133301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32117f1-e755-4a17-b7b0-dd74e64c7480 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.190839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.191596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.191775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.192049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.193546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.194159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.194441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.194714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.199567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:00.208815 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:00.213898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:00.214741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 51/108] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:00.215912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.217775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:00.220647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:00.222647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:00.225244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 58/109] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:47:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:00.227350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.345664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b3584-c55e-408a-8974-eeae9671f681 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.354609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abef586-7ec6-4c77-a458-33e8dca9088a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.382370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.382959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.383241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.383524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.384687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:00.386160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:00.387831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 52/110] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:00.388550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.400876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.401801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.402236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.402516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.403181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:00.403484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:00.404882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:00.413437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 59/111] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:00.417188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.461495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:00.486033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:00.495926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcdbf1-5726-43ed-9ea7-b0167538d5d1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.527892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fdc18a-0795-4b43-a179-b4e2a9e797cf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.532166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.532784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.533067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.533340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.534013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:00.534297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:00.535682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:00.544008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 53/112] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:00.544837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.565052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.565683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.565958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.566226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.576096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:00.581305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:00.582154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 60/113] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:00.583662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.647128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a0a08-1f70-416b-94e0-faceaee2f1f1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.680564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.681192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.681472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.681743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.681941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd3b1-b01b-4cb4-9f66-183662ef2ed2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.683676 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:00.693898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:00.698965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:00.699118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:00.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 54/114] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:00.706070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.721367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5577f6e-7694-44ad-ac40-4ddb65f5deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.721980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5577f6e-7694-44ad-ac40-4ddb65f5deeb None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.722248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5577f6e-7694-44ad-ac40-4ddb65f5deeb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.722511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5577f6e-7694-44ad-ac40-4ddb65f5deeb None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.723181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5577f6e-7694-44ad-ac40-4ddb65f5deeb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:00.724777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5577f6e-7694-44ad-ac40-4ddb65f5deeb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:00.751899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 61/115] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:48:00 2022] POST /identity/v3/projects => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:00.753154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb3663-f413-470c-ae2f-009c4e190fc6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.753440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb3663-f413-470c-ae2f-009c4e190fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.753713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb3663-f413-470c-ae2f-009c4e190fc6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.755006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 62/116] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:00.755736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb3663-f413-470c-ae2f-009c4e190fc6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.822873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77716c9-290e-476f-a31d-3346c7944dfa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.861205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.861869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.862154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.862430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] PATH_INFO: `/v3/users/3e9d72b8027840c5ac30b87d38ea3920` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.877409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] RBAC: Authorizing `identity:get_user(user_id=3e9d72b8027840c5ac30b87d38ea3920)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:00.878954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:00.881076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 55/117] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/users/3e9d72b8027840c5ac30b87d38ea3920 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:00.881897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.897205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb3663-f413-470c-ae2f-009c4e190fc6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:00.937803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:00.938451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:00.938732 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:00.939003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:00.940127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:00.941704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:00.943834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 63/118] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:00.944654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:00.951834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:00.986750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:01.005059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f91905-b1cf-47d2-9a10-9a55fd0e73ed None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.042788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a82c76-2b6c-4629-b9c1-ef128981f09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.043417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a82c76-2b6c-4629-b9c1-ef128981f09c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.043749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a82c76-2b6c-4629-b9c1-ef128981f09c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.044031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a82c76-2b6c-4629-b9c1-ef128981f09c None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.044727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a82c76-2b6c-4629-b9c1-ef128981f09c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:01.045033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a82c76-2b6c-4629-b9c1-ef128981f09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.046444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a82c76-2b6c-4629-b9c1-ef128981f09c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.050971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b293d9-6db7-4c7c-ab02-f468fb9ef0a0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.055108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 56/119] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:01.060664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d55bd-db84-4768-8f6d-cb40e771979e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.060955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d55bd-db84-4768-8f6d-cb40e771979e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.061709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d55bd-db84-4768-8f6d-cb40e771979e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.085343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-173d55bd-db84-4768-8f6d-cb40e771979e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:01.086536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-173d55bd-db84-4768-8f6d-cb40e771979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:01.087116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.087739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.088024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.088335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.088998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:01.089303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.090669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.103803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 64/120] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:01.103803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.168935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 57/121] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:01.170021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d55bd-db84-4768-8f6d-cb40e771979e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.217143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:01.221050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1117dd-7534-4ce7-be2e-3883b3f36318 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.236454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:01.262666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.264060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.264810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.265267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.303670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d55bd-db84-4768-8f6d-cb40e771979e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.350124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.351689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.352087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.352490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.354367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.356316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.363453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 58/122] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:01.363453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.364095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.366316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.488711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b998df4-07c5-49ef-bad0-06adb14b00a5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.498952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:01.503114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 65/123] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:01.504282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.508081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:01.531849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.532610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.532904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.533179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.534322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.536105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.542241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 59/124] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:01.542241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.613122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304f5c8-ae46-4b29-b7d9-f366f9fdf61d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.641350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef5706-3439-4f2e-aef5-de8c9abb4b2d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.649097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.649738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.650016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.650290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] PATH_INFO: `/v3/users/fce1040c54ac43928ab07aa4e6b6d613` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.651408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=fce1040c54ac43928ab07aa4e6b6d613)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.652888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.658764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 66/125] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/users/fce1040c54ac43928ab07aa4e6b6d613 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:01.658764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.682155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.682835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.683118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.683398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3/users/e46173bc1d274512ae45ef02806797bc/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.685342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] RBAC: Authorizing `identity:create_grant(project_id=8567939112d5422e96682571a1014ec3, user_id=e46173bc1d274512ae45ef02806797bc, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.686800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.708216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 60/126] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:01 2022] PUT /identity/v3/projects/8567939112d5422e96682571a1014ec3/users/e46173bc1d274512ae45ef02806797bc/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:01.708216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.749385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:01.760007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:01.806340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6cbc3-2413-45df-8588-9d472eeed9d7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.828321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de753e-0176-498c-a9c3-5234a21e5468 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:01.855093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.858299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.858299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.858299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.870828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:01.872331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:01.873260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:01.873793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:01.874270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:01.875445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:01.877530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:01.877638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:01.878886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:01.880275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 67/127] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:01.881979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 61/128] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:01.882045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:01.882108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:02.005538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bedb8c-c5a3-418d-8723-46692c076668 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.018424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9ae93-fed7-4967-a7c1-771b62c31478 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.032996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:02.041678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:02.042414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.043061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.043339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.043934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.045290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.046910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.048564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 68/129] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:02.049333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:02.063988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.064969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.065483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.066358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.068484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.070602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.073253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 62/130] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:01 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:02.074466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:02.164338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f6c0f-c264-4198-b3da-436260821241 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.198911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.199530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.199811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.200083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.200800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:02.201104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.201360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e813b66-310b-4061-8082-9f2edf1f7bb1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.202500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.211834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 69/131] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:02.212692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:02.239889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.241269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.241734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.242317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.243377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] RBAC: Adding query filter params (role.id=e9db72a727fb4cd1a83a7cd4965c51a3, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=3e9d72b8027840c5ac30b87d38ea3920) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:02.243895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.245814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.252311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 63/132] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:02 2022] GET /identity/v3/role_assignments?user.id=3e9d72b8027840c5ac30b87d38ea3920&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=e9db72a727fb4cd1a83a7cd4965c51a3 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:02.253360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:02.296561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:02.303163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:02.370586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cdd2b-0704-4140-a8c5-4586ad0e8b18 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.380539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2057d-aa08-4c5a-af3f-ddf0b0f31bf5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.412505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.413601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.414072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.414535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3/users/fce1040c54ac43928ab07aa4e6b6d613/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.416975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] RBAC: Authorizing `identity:create_grant(project_id=8567939112d5422e96682571a1014ec3, user_id=fce1040c54ac43928ab07aa4e6b6d613, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.419652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.439546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 64/133] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:02 2022] PUT /identity/v3/projects/8567939112d5422e96682571a1014ec3/users/fce1040c54ac43928ab07aa4e6b6d613/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:02.440767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:02.469907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.470257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.470481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.470753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3/users/e46173bc1d274512ae45ef02806797bc/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.478291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=8567939112d5422e96682571a1014ec3, user_id=e46173bc1d274512ae45ef02806797bc, role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.479799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.515786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 70/134] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:02 2022] PUT /identity/v3/projects/8567939112d5422e96682571a1014ec3/users/e46173bc1d274512ae45ef02806797bc/roles/18b82b06d87e4c93bef42bca94fc658d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:02.567136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:02.571117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea622c-ad75-4a1c-b2f8-6fc97e6e0cab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.611962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec989f47-3dc1-470d-a1af-ec4d2d895a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.613051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989f47-3dc1-470d-a1af-ec4d2d895a77 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.613533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989f47-3dc1-470d-a1af-ec4d2d895a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.613998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989f47-3dc1-470d-a1af-ec4d2d895a77 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.619931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec989f47-3dc1-470d-a1af-ec4d2d895a77 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.621968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec989f47-3dc1-470d-a1af-ec4d2d895a77 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.624420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 65/135] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:02 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:02.630117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:02.705255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065121a-57a3-42f0-a831-e5106ce5cc10 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.705607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065121a-57a3-42f0-a831-e5106ce5cc10 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.705821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065121a-57a3-42f0-a831-e5106ce5cc10 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.707415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 66/136] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:02.713463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6601-9b26-4f84-be16-d5bf03bf5747 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.713746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6601-9b26-4f84-be16-d5bf03bf5747 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.714022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6601-9b26-4f84-be16-d5bf03bf5747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.759715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a5eb6601-9b26-4f84-be16-d5bf03bf5747 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:02.760378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a5eb6601-9b26-4f84-be16-d5bf03bf5747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:02.806311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae18d0-44d7-461e-ab9c-4cb83723f1a6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:02.828131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:02.867679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:02.868760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:02.869241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:02.869708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] PATH_INFO: `/v3/projects/a66dafb589fd43558a1aa89dd6e16031` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:02.872854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=a66dafb589fd43558a1aa89dd6e16031)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:02.874510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:02.874985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 67/137] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:02.881164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 71/138] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:02 2022] GET /identity/v3/projects/a66dafb589fd43558a1aa89dd6e16031 => generated 312 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:02.887084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6601-9b26-4f84-be16-d5bf03bf5747 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:03.008391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:03.012011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6601-9b26-4f84-be16-d5bf03bf5747 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:03.058319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:03.059086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:03.059369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:03.059701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] PATH_INFO: `/v3/projects/a66dafb589fd43558a1aa89dd6e16031/users/e46173bc1d274512ae45ef02806797bc/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:03.061792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] RBAC: Authorizing `identity:create_grant(project_id=a66dafb589fd43558a1aa89dd6e16031, user_id=e46173bc1d274512ae45ef02806797bc, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:03.063313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:03.091553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 68/139] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:02 2022] PUT /identity/v3/projects/a66dafb589fd43558a1aa89dd6e16031/users/e46173bc1d274512ae45ef02806797bc/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:03.447190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:03.559161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532aeabf-01b5-4bce-b036-4317fc5c6fc3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:03.592547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e40d1-eae6-40fe-99c5-a9faa761da75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:03.593276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e40d1-eae6-40fe-99c5-a9faa761da75 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:03.593561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e40d1-eae6-40fe-99c5-a9faa761da75 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:03.593839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e40d1-eae6-40fe-99c5-a9faa761da75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:03.595003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e40d1-eae6-40fe-99c5-a9faa761da75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:03.596591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e40d1-eae6-40fe-99c5-a9faa761da75 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:03.598178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 72/140] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:03.604135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:03.732363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98519e-ea18-48e2-8703-2da569c89e7d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:03.785958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db077344-687d-4d99-99ac-8c8daed47688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:03.786633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077344-687d-4d99-99ac-8c8daed47688 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:03.786906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077344-687d-4d99-99ac-8c8daed47688 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:03.787185 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db077344-687d-4d99-99ac-8c8daed47688 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:03.787882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db077344-687d-4d99-99ac-8c8daed47688 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:03.789382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db077344-687d-4d99-99ac-8c8daed47688 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:03.825337 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 69/141] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:03 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:04.268645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ece78-07c5-430b-809e-0388f9be1b79 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:04.269161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ece78-07c5-430b-809e-0388f9be1b79 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:04.269528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ece78-07c5-430b-809e-0388f9be1b79 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:04.271404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 73/142] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:04.282108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5601c6-237b-451d-8169-884d1a82c3ca None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:04.282413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5601c6-237b-451d-8169-884d1a82c3ca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:04.282679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5601c6-237b-451d-8169-884d1a82c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:04.305149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1f5601c6-237b-451d-8169-884d1a82c3ca None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:04.306339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1f5601c6-237b-451d-8169-884d1a82c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:04.320589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:04.406655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 70/143] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:04.933073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ece78-07c5-430b-809e-0388f9be1b79 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:05.000682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:05.050933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ece78-07c5-430b-809e-0388f9be1b79 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:05.095899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171311e8-17be-4bdd-a32d-07dd507b90fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:05.096574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171311e8-17be-4bdd-a32d-07dd507b90fe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.096856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171311e8-17be-4bdd-a32d-07dd507b90fe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.097128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171311e8-17be-4bdd-a32d-07dd507b90fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.110236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171311e8-17be-4bdd-a32d-07dd507b90fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-171311e8-17be-4bdd-a32d-07dd507b90fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:05.115563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:05.116473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:05.117271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 74/144] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:05.123836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5601c6-237b-451d-8169-884d1a82c3ca None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:05.195547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5688da-2826-4aff-96bc-4864af378a42 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.195802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5688da-2826-4aff-96bc-4864af378a42 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.196509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5688da-2826-4aff-96bc-4864af378a42 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.198765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 75/145] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:05.204902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ed387-ee46-4cb2-87e4-7ce8e2931b6f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.205767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ed387-ee46-4cb2-87e4-7ce8e2931b6f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.206124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ed387-ee46-4cb2-87e4-7ce8e2931b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.239462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-733ed387-ee46-4cb2-87e4-7ce8e2931b6f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:05.241191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-733ed387-ee46-4cb2-87e4-7ce8e2931b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:05.260890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5601c6-237b-451d-8169-884d1a82c3ca None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:05.324081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3fe150-6a4a-4e96-af23-6301ef818dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:05.324854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fe150-6a4a-4e96-af23-6301ef818dbf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.325133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fe150-6a4a-4e96-af23-6301ef818dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.325402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3fe150-6a4a-4e96-af23-6301ef818dbf None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.326070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fe150-6a4a-4e96-af23-6301ef818dbf None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:05.326368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fe150-6a4a-4e96-af23-6301ef818dbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:05.328417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3fe150-6a4a-4e96-af23-6301ef818dbf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:05.347989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 71/146] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:05.350276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689b8a-4c95-4d15-ab10-fd2afdd95c8b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.350720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689b8a-4c95-4d15-ab10-fd2afdd95c8b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.351141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3689b8a-4c95-4d15-ab10-fd2afdd95c8b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.352642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 72/147] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:05.354139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f9d3-0d56-48bf-9fa3-85f57f456498 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.354681 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f9d3-0d56-48bf-9fa3-85f57f456498 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.355145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f9d3-0d56-48bf-9fa3-85f57f456498 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.356894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 73/148] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:05.357805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4f9d3-0d56-48bf-9fa3-85f57f456498 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:05.371688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 76/149] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:05.373963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1364a-e772-44fc-bc6b-b92c61f801cb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.373963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1364a-e772-44fc-bc6b-b92c61f801cb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.373963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1364a-e772-44fc-bc6b-b92c61f801cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.392547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:05.394170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0ff1364a-e772-44fc-bc6b-b92c61f801cb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:05.395420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0ff1364a-e772-44fc-bc6b-b92c61f801cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:05.444063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:05.476888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 77/150] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:05.478083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ac41-e46d-49ee-a5bc-16f1143dabb4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.478369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ac41-e46d-49ee-a5bc-16f1143dabb4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.478647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098ac41-e46d-49ee-a5bc-16f1143dabb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.516061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6098ac41-e46d-49ee-a5bc-16f1143dabb4 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:05.517364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6098ac41-e46d-49ee-a5bc-16f1143dabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:05.585580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4f9d3-0d56-48bf-9fa3-85f57f456498 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:05.614032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 78/151] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:05.645814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:05.646482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.646763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.647041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.648223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:05.649776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:05.651689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 74/152] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:05 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:05.657130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098ac41-e46d-49ee-a5bc-16f1143dabb4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:05.716610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:05.784058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098ac41-e46d-49ee-a5bc-16f1143dabb4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:05.797061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:05.832174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:05.832174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:05.832174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:05.832174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:05.845303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:05.905391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd2e5d-9280-47e2-8abf-4e3126786c93 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:05.963010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:05.964545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:06.001478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:06.002513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.002998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.003473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.033113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:06.039271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:06.040127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:06.040989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:06.043275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 75/153] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:06.053306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:06.156943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 79/154] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:06.163058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:06.168470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:06.294245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4f27e-0932-4d38-b88a-cdf2fa1038ef None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:06.354546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe5dbd-ae83-4081-9d6e-6e3fa5226af3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:06.398294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:06.399007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.399298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.399576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.400294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:06.400602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:06.402056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:06.403753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:06.414862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:06.415518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.415800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.416075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.418807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:06.420314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:06.424311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 76/156] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:06.425090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:06.427745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 80/156] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:06.428604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:06.520565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:06.572699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc989e0-a107-4f73-ab2e-b4adb12a7333 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:06.630788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a3019-534e-482b-96de-c6ffa8bbb299 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:06.642467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cea87-9fa1-4df1-b18d-359f50b55fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:06.643126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cea87-9fa1-4df1-b18d-359f50b55fae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.643410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cea87-9fa1-4df1-b18d-359f50b55fae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.643684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cea87-9fa1-4df1-b18d-359f50b55fae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.661192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cea87-9fa1-4df1-b18d-359f50b55fae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f38cea87-9fa1-4df1-b18d-359f50b55fae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:06.667265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:06.668364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:06.669212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:06.671178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 81/157] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:06.672648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553bbe5-b1c4-4197-b16a-a5073325ec2c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.673195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553bbe5-b1c4-4197-b16a-a5073325ec2c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.673681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553bbe5-b1c4-4197-b16a-a5073325ec2c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.675118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 82/158] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:06.676004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553bbe5-b1c4-4197-b16a-a5073325ec2c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:06.698413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:06.699061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.699340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.699615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.720527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:06.733458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:06.734349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:06.735201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 77/159] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:06.735855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:06.790553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553bbe5-b1c4-4197-b16a-a5073325ec2c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:06.797134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:06.825072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:06.830759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:06.831647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.832370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.832920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.834165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=e46173bc1d274512ae45ef02806797bc) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:06.834687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:06.836478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:06.843660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 83/160] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/role_assignments?user.id=e46173bc1d274512ae45ef02806797bc&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:06.844826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:06.846837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06377aa-176e-4fa5-8d06-70d0ce00aa9d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:06.887927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956feb0-8301-4924-ae29-f6384790e3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:06.888612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956feb0-8301-4924-ae29-f6384790e3cd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.888899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956feb0-8301-4924-ae29-f6384790e3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.889181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956feb0-8301-4924-ae29-f6384790e3cd None admin] PATH_INFO: `/v3/users/fce1040c54ac43928ab07aa4e6b6d613` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.890322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956feb0-8301-4924-ae29-f6384790e3cd None admin] RBAC: Authorizing `identity:get_user(user_id=fce1040c54ac43928ab07aa4e6b6d613)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:06.891800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956feb0-8301-4924-ae29-f6384790e3cd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:06.893595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 78/161] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/users/fce1040c54ac43928ab07aa4e6b6d613 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:06.899689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d0c30-aa3b-4843-adb6-f423b05f90be None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:06.899988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d0c30-aa3b-4843-adb6-f423b05f90be None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:06.900278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d0c30-aa3b-4843-adb6-f423b05f90be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:06.921710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e99d0c30-aa3b-4843-adb6-f423b05f90be None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:06.922921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e99d0c30-aa3b-4843-adb6-f423b05f90be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:06.964331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27615707-5980-4a0e-8d6b-5a42ce93c6ca None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:06.998049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 79/162] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:07.004015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.005059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.005260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d0c30-aa3b-4843-adb6-f423b05f90be None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.005898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.006366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.007238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:07.007900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.009603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.019211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 84/163] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:07.020705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.086227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:07.086852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:07.136388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf4a31-8ed5-43ed-bb53-57975edf5afe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.151302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d0c30-aa3b-4843-adb6-f423b05f90be None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.174217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.175241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.175717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.176259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.188171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.188475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.188697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.188979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.189648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:07.189950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.191365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.204177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 80/164] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:07.204177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.278843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.280694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.320818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fddac-d8fc-4ba0-b060-2232c3ab6657 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.343395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:07.353391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:07.389229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.389840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.390125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.390402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.391672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.393242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.395013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 81/165] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:07.397024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.401990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 85/166] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:07.403131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.508195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef62ed-8109-4250-9640-50efdba33206 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.544971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.546055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.546531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.546994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.565986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da803e04-abdd-4258-a4a5-386d6299cc83 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.577637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:07.622454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.624286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.627947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.628491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.628778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.629056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] PATH_INFO: `/v3/users/3e9d72b8027840c5ac30b87d38ea3920` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.632879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=3e9d72b8027840c5ac30b87d38ea3920)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.634373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.636183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 82/167] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/users/3e9d72b8027840c5ac30b87d38ea3920 => generated 289 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:07.637011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.726834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:07.738810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 86/168] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:07.740348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.799338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac78a566-4545-4005-b304-60224f165ba9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.804712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:07.841666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.842411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.842698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.842975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.844231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.845707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.847913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 83/169] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:07.848844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.850613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b76f46-8935-48ac-905c-e0049953fde2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.889043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:07.890021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:07.890506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:07.891014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:07.892484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:07.894474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:07.896810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 87/170] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:07.897939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:07.968225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a55cd-27e4-4c46-8b85-6bf5a7a98377 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:07.990161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:08.006378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5427b9-7b63-4e2a-af49-10a8efc0a346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.007026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5427b9-7b63-4e2a-af49-10a8efc0a346 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.007310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5427b9-7b63-4e2a-af49-10a8efc0a346 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.007587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5427b9-7b63-4e2a-af49-10a8efc0a346 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.009651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c056e68-79a2-4e5e-ad61-2deba6671722 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:08.019050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5427b9-7b63-4e2a-af49-10a8efc0a346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ad5427b9-7b63-4e2a-af49-10a8efc0a346 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:08.026213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:08.027110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:08.027977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:08.032207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 84/171] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:08.033645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e32b53-00d9-46cb-b99b-f7bb03f04faa None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.033929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e32b53-00d9-46cb-b99b-f7bb03f04faa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.034268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e32b53-00d9-46cb-b99b-f7bb03f04faa None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.036021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 85/172] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:08.041066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e32b53-00d9-46cb-b99b-f7bb03f04faa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:08.042568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:08.052572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.053558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.054044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.054510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.055975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:08.057871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:08.059843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 88/173] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:07 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:08.060873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:08.157757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e32b53-00d9-46cb-b99b-f7bb03f04faa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:08.173446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3582e2-77c9-4e22-bdc4-341c162618a0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:08.192892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.193660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.194032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.194371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.195156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=fce1040c54ac43928ab07aa4e6b6d613) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:08.195523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:08.197071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:08.203905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 86/174] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:08 2022] GET /identity/v3/role_assignments?user.id=fce1040c54ac43928ab07aa4e6b6d613&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:08.204756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:08.209042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969adbd6-eb0b-4383-bc6e-540557fd90ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.209712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969adbd6-eb0b-4383-bc6e-540557fd90ad None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.209989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969adbd6-eb0b-4383-bc6e-540557fd90ad None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.210259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969adbd6-eb0b-4383-bc6e-540557fd90ad None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3/users/3e9d72b8027840c5ac30b87d38ea3920/roles/e9db72a727fb4cd1a83a7cd4965c51a3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.216816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969adbd6-eb0b-4383-bc6e-540557fd90ad None admin] RBAC: Authorizing `identity:create_grant(project_id=8567939112d5422e96682571a1014ec3, user_id=3e9d72b8027840c5ac30b87d38ea3920, role_id=e9db72a727fb4cd1a83a7cd4965c51a3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:08.216816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969adbd6-eb0b-4383-bc6e-540557fd90ad None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:08.234203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 89/175] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:08 2022] PUT /identity/v3/projects/8567939112d5422e96682571a1014ec3/users/3e9d72b8027840c5ac30b87d38ea3920/roles/e9db72a727fb4cd1a83a7cd4965c51a3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:08.236386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a19771-4bb5-481f-8e8a-b72a2d8ddbdb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.237023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a19771-4bb5-481f-8e8a-b72a2d8ddbdb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.237396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a19771-4bb5-481f-8e8a-b72a2d8ddbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.262939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-30a19771-4bb5-481f-8e8a-b72a2d8ddbdb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:08.265004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-30a19771-4bb5-481f-8e8a-b72a2d8ddbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:08.269044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:08.338591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:08.354940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07888772-c58a-4d79-9e62-e11ec5e2f2df None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:08.365620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 90/176] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:08.366694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a19771-4bb5-481f-8e8a-b72a2d8ddbdb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:08.451457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0cb6-91e4-4b53-891a-8d5e0d99e013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.452005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0cb6-91e4-4b53-891a-8d5e0d99e013 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.452313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0cb6-91e4-4b53-891a-8d5e0d99e013 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.452584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0cb6-91e4-4b53-891a-8d5e0d99e013 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.453264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0cb6-91e4-4b53-891a-8d5e0d99e013 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:08.453560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0cb6-91e4-4b53-891a-8d5e0d99e013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:08.455024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0cb6-91e4-4b53-891a-8d5e0d99e013 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:08.468351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 87/177] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:08.470278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678c7c-ad43-46e1-ad34-ea054a6abe8b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.470767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678c7c-ad43-46e1-ad34-ea054a6abe8b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.471409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678c7c-ad43-46e1-ad34-ea054a6abe8b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.473273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 88/178] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:08.474307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678c7c-ad43-46e1-ad34-ea054a6abe8b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:08.544704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a19771-4bb5-481f-8e8a-b72a2d8ddbdb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:08.584349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6a6ec-5396-411b-a4c3-fd76f139fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.585361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6a6ec-5396-411b-a4c3-fd76f139fa79 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.585869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6a6ec-5396-411b-a4c3-fd76f139fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.586436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6a6ec-5396-411b-a4c3-fd76f139fa79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.587474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6a6ec-5396-411b-a4c3-fd76f139fa79 None admin] RBAC: Adding query filter params (role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=e46173bc1d274512ae45ef02806797bc) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:08.587975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6a6ec-5396-411b-a4c3-fd76f139fa79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:08.590080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6a6ec-5396-411b-a4c3-fd76f139fa79 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:08.597029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 91/179] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:08 2022] GET /identity/v3/role_assignments?user.id=e46173bc1d274512ae45ef02806797bc&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:08.597542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678c7c-ad43-46e1-ad34-ea054a6abe8b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:08.598910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45e123-664d-4339-88b1-0dbb149fe8b0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.599489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45e123-664d-4339-88b1-0dbb149fe8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.599996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45e123-664d-4339-88b1-0dbb149fe8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.605918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:08.624281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ec45e123-664d-4339-88b1-0dbb149fe8b0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:08.625909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ec45e123-664d-4339-88b1-0dbb149fe8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:08.640068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.640767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.641043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.641311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] PATH_INFO: `/v3/users/e46173bc1d274512ae45ef02806797bc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.642546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] RBAC: Authorizing `identity:get_user(user_id=e46173bc1d274512ae45ef02806797bc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:08.644008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:08.646582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 89/180] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:08 2022] GET /identity/v3/users/e46173bc1d274512ae45ef02806797bc => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:08.652779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:08.700562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:08.755099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 92/181] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:08.782296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c086-137b-423b-94ec-3d57e4eec562 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:08.826963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:08.827596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:08.827872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:08.828141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:08.897466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:08.938072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:08.939564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:08.948858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45e123-664d-4339-88b1-0dbb149fe8b0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.054905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 90/182] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 16:48:09.062178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.109614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45e123-664d-4339-88b1-0dbb149fe8b0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.144345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:09.152138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198196df-a6b9-4cb5-bd9d-ffc8841aa218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.152800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198196df-a6b9-4cb5-bd9d-ffc8841aa218 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.153079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198196df-a6b9-4cb5-bd9d-ffc8841aa218 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.153349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198196df-a6b9-4cb5-bd9d-ffc8841aa218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.171024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198196df-a6b9-4cb5-bd9d-ffc8841aa218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-198196df-a6b9-4cb5-bd9d-ffc8841aa218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:09.175943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:09.177023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:09.177793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 93/183] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:09.178440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4bf33-8e95-42e1-89b7-f5453d4c3cc4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.178965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de5607-594b-475c-a170-a92ade5ee965 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.179249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de5607-594b-475c-a170-a92ade5ee965 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.179404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de5607-594b-475c-a170-a92ade5ee965 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.181336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 94/184] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:09.182059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de5607-594b-475c-a170-a92ade5ee965 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.221773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.222538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.222874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.223199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] PATH_INFO: `/v3/projects/a66dafb589fd43558a1aa89dd6e16031` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.224492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] RBAC: Authorizing `identity:get_project(project_id=a66dafb589fd43558a1aa89dd6e16031)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:09.226082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:09.227901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 91/185] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/projects/a66dafb589fd43558a1aa89dd6e16031 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:09.233846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.241483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:09.308502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de5607-594b-475c-a170-a92ade5ee965 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.334755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174a172-f87b-4ed6-971e-0a9ca70605e0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.344785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f813ba5-e5b5-45ff-8a48-4c786fcfb868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.345722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f813ba5-e5b5-45ff-8a48-4c786fcfb868 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.346006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f813ba5-e5b5-45ff-8a48-4c786fcfb868 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.346317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f813ba5-e5b5-45ff-8a48-4c786fcfb868 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.357847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f813ba5-e5b5-45ff-8a48-4c786fcfb868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6f813ba5-e5b5-45ff-8a48-4c786fcfb868 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:09.363282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:09.364636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:09.365435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 95/186] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:09.366539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea880c4-9b33-4963-943f-76cde641bfe9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.366539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea880c4-9b33-4963-943f-76cde641bfe9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.366826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea880c4-9b33-4963-943f-76cde641bfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.381088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.381717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.381999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.382275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.382952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:09.383255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:09.384685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:09.390512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2ea880c4-9b33-4963-943f-76cde641bfe9 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:09.391879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2ea880c4-9b33-4963-943f-76cde641bfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:09.394418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 92/187] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:09.395426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.398937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:09.463420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 96/188] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:09.464301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea880c4-9b33-4963-943f-76cde641bfe9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.505965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:09.511593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe4bd5-246f-4dae-bcbb-a103750673ab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.549793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.550927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.551603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.552081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.553263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=a66dafb589fd43558a1aa89dd6e16031, user.id=e46173bc1d274512ae45ef02806797bc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:09.553785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:09.555564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:09.568294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 93/189] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/role_assignments?user.id=e46173bc1d274512ae45ef02806797bc&scope.project.id=a66dafb589fd43558a1aa89dd6e16031&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:09.569836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.592919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea880c4-9b33-4963-943f-76cde641bfe9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.639643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.640440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.640807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.642153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.642153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:09.642509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:09.644948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:09.654511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 97/190] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:09.662151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.711881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:09.762186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f43f91-aba6-4a75-8390-3f3df506c764 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.773172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:09.786152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b772f-30f0-4f04-8400-a2e676040639 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.812436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.813332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.813612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.813883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] PATH_INFO: `/v3/users/345731bf46dd414d8501dd898ceaf6a3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.827951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.828631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.828910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.829314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] PATH_INFO: `/v3/users/345731bf46dd414d8501dd898ceaf6a3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.830985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] RBAC: Authorizing `identity:get_user(user_id=345731bf46dd414d8501dd898ceaf6a3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:09.831405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] RBAC: Authorizing `identity:get_user(user_id=345731bf46dd414d8501dd898ceaf6a3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:09.833059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:09.833880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:09.835343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 94/191] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/users/345731bf46dd414d8501dd898ceaf6a3 => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:09.835602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 98/192] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/users/345731bf46dd414d8501dd898ceaf6a3 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:09.839844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.840976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.941400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c5840-f39b-40af-8ad8-db4f15a44e27 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.953194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb4b08-c1f6-4258-9d7a-e4689874d9e9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:09.977841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.978699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.978922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.979253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:09.980788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:09.982296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:09.982790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:09.984307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 99/193] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:09.985211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:09.996904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:09.997525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:09.997806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:09.998282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.014188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:10.072945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:10.074572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:10.092379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11542046-ebf2-4a00-93cd-d984fc7be91b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:10.130733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:10.131372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.131624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.131901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.183822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 95/194] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:10.189137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:10.204844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:10.206327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:10.217749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:10.233634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:10.291887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd1d69-818a-4edb-a411-7e70e5c2ef1a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:10.309389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 100/195] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:10.314949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:10.329068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:10.329686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.329966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.330225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.330889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:10.332403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:10.355913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 96/196] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:10 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:10.356733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:10.443524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf45b9-0c7d-4354-8991-e36b4172f494 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:10.471931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:10.481503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5e8bd-2253-4cbc-9fe9-8bf958aed5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:10.482163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e8bd-2253-4cbc-9fe9-8bf958aed5bd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.482440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e8bd-2253-4cbc-9fe9-8bf958aed5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.482711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5e8bd-2253-4cbc-9fe9-8bf958aed5bd None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.483881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5e8bd-2253-4cbc-9fe9-8bf958aed5bd None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:10.485397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5e8bd-2253-4cbc-9fe9-8bf958aed5bd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:10.487123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 101/197] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:10 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:10.488363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b6a99-0af2-44ea-8471-35e440085f82 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.488669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b6a99-0af2-44ea-8471-35e440085f82 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.488995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b6a99-0af2-44ea-8471-35e440085f82 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.490338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 102/198] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:10.493288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b6a99-0af2-44ea-8471-35e440085f82 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:10.500959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:10.525929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a43ffe-280a-4c82-9030-eeb462d51486 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:10.564525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50d5bf-f8d6-472b-9e54-3d913f65b6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:10.565179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50d5bf-f8d6-472b-9e54-3d913f65b6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.565466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50d5bf-f8d6-472b-9e54-3d913f65b6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.565740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50d5bf-f8d6-472b-9e54-3d913f65b6f6 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.569770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50d5bf-f8d6-472b-9e54-3d913f65b6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:10.571236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50d5bf-f8d6-472b-9e54-3d913f65b6f6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:10.572978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 97/199] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:10 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:10.577951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4d3e8-5dbc-43eb-a59c-276487373869 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.577951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4d3e8-5dbc-43eb-a59c-276487373869 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.577951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4d3e8-5dbc-43eb-a59c-276487373869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.593975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-92c4d3e8-5dbc-43eb-a59c-276487373869 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:10.595174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-92c4d3e8-5dbc-43eb-a59c-276487373869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:10.625371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b6a99-0af2-44ea-8471-35e440085f82 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:10.667312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:10.667989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.668305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.668585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.669317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=345731bf46dd414d8501dd898ceaf6a3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:10.669630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:10.671661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:10.677983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 103/200] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:10 2022] GET /identity/v3/role_assignments?user.id=345731bf46dd414d8501dd898ceaf6a3&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:10.681127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:10.682888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 98/201] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:10.756299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:10.804408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2bd0e-4429-4f18-b786-19a54c8263f7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:10.842089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:10.842705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:10.842982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:10.843252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:10.843986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] RBAC: Adding query filter params (role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=345731bf46dd414d8501dd898ceaf6a3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:10.844326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:10.845828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:10.851737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 104/202] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:10 2022] GET /identity/v3/role_assignments?user.id=345731bf46dd414d8501dd898ceaf6a3&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:10.971933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4d3e8-5dbc-43eb-a59c-276487373869 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:11.055658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:11.075364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4d3e8-5dbc-43eb-a59c-276487373869 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:11.116064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:11.116744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:11.117023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:11.117297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:11.127564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:11.132887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:11.133775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:11.134559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 99/203] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:11.139862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:11.273217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e1e4-1dd0-4df8-a3a8-6c25947afc06 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:11.296641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:11.308681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:11.309301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:11.309578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:11.309845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:11.310511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:11.310812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:11.312195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:11.321206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 105/204] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:11.329454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:11.473887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cc8b5-8244-4113-96c1-f311a48f13e9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:11.509257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:11.510659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:11.511107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:11.511547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] PATH_INFO: `/v3/users/3e9d72b8027840c5ac30b87d38ea3920` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:11.512869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] RBAC: Authorizing `identity:get_user(user_id=3e9d72b8027840c5ac30b87d38ea3920)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:11.514535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:11.516417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 100/205] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:11 2022] GET /identity/v3/users/3e9d72b8027840c5ac30b87d38ea3920 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:11.525682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:11.655116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca9535-383d-40e6-b0b4-3c84049cabe2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:11.693392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:11.694343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:11.694806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:11.695247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:11.749469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:11.769339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:11.771807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:11.881742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 106/206] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:11.889363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:11.902810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:11.996038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87de8f-3c42-495c-ae20-8d0a9bcc2343 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:12.033820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b741f87-c316-461c-a95e-b28300432686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:12.034844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b741f87-c316-461c-a95e-b28300432686 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:12.035318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b741f87-c316-461c-a95e-b28300432686 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:12.035830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b741f87-c316-461c-a95e-b28300432686 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:12.037734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b741f87-c316-461c-a95e-b28300432686 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:12.039402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b741f87-c316-461c-a95e-b28300432686 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:12.041359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 101/207] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:11 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:12.049458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:12.178155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:12.194555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800be96-81cd-468d-a5ef-a1ad723cc682 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:12.230261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8f5c3-4a97-4285-a680-729c1a2e5242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:12.231183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f5c3-4a97-4285-a680-729c1a2e5242 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:12.231459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f5c3-4a97-4285-a680-729c1a2e5242 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:12.231784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f5c3-4a97-4285-a680-729c1a2e5242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:12.232541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f5c3-4a97-4285-a680-729c1a2e5242 None admin] RBAC: Adding query filter params (role.id=e9db72a727fb4cd1a83a7cd4965c51a3, scope.project.id=8567939112d5422e96682571a1014ec3, user.id=3e9d72b8027840c5ac30b87d38ea3920) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:12.232846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f5c3-4a97-4285-a680-729c1a2e5242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:12.234350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f5c3-4a97-4285-a680-729c1a2e5242 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:12.241019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 107/208] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:12 2022] GET /identity/v3/role_assignments?user.id=3e9d72b8027840c5ac30b87d38ea3920&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=e9db72a727fb4cd1a83a7cd4965c51a3 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:12.797949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63acab8-356d-48a2-807a-813bf7b72fd5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:12.798467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63acab8-356d-48a2-807a-813bf7b72fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:12.798467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63acab8-356d-48a2-807a-813bf7b72fd5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:12.800024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 102/209] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:12.805705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203d851-3821-434c-afc4-ca562972b2f7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:12.806246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203d851-3821-434c-afc4-ca562972b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:12.806718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203d851-3821-434c-afc4-ca562972b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:12.825602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d203d851-3821-434c-afc4-ca562972b2f7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:12.827131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d203d851-3821-434c-afc4-ca562972b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:12.904587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 108/210] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:13.041204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b835d-dffe-4ef0-9950-e804a0ae5e29 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.041530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b835d-dffe-4ef0-9950-e804a0ae5e29 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.041820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b835d-dffe-4ef0-9950-e804a0ae5e29 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.043346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 103/211] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:13.048667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef837e7-6273-4fe3-ade1-d367b015a8a3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.048888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef837e7-6273-4fe3-ade1-d367b015a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.049160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef837e7-6273-4fe3-ade1-d367b015a8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.050036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb518-995a-4e4a-a1b4-a9913a626e7d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.050526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb518-995a-4e4a-a1b4-a9913a626e7d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.050989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb518-995a-4e4a-a1b4-a9913a626e7d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.053519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 109/212] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:13.061444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c32d7-02e7-4fcd-b923-8d7956a9d39f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.061725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c32d7-02e7-4fcd-b923-8d7956a9d39f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.061996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c32d7-02e7-4fcd-b923-8d7956a9d39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.071061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cef837e7-6273-4fe3-ade1-d367b015a8a3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:13.072560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cef837e7-6273-4fe3-ade1-d367b015a8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:13.081779 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-262c32d7-02e7-4fcd-b923-8d7956a9d39f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:13.083105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-262c32d7-02e7-4fcd-b923-8d7956a9d39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:13.164190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 104/213] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:13.164396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 110/214] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:13.180260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef837e7-6273-4fe3-ade1-d367b015a8a3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.191017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:13.261898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94abe8-ad6f-4e85-9d45-a8640320cc66 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.262420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94abe8-ad6f-4e85-9d45-a8640320cc66 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.262420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94abe8-ad6f-4e85-9d45-a8640320cc66 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.263886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 111/215] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:13.268877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b14d-d61c-449d-abf5-7facd252bbd3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.269164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b14d-d61c-449d-abf5-7facd252bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.269427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b14d-d61c-449d-abf5-7facd252bbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.288871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6436b14d-d61c-449d-abf5-7facd252bbd3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:13.290053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6436b14d-d61c-449d-abf5-7facd252bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:13.292419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:13.303898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef837e7-6273-4fe3-ade1-d367b015a8a3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.340838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.341590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.341977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.342310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.353537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:13.358870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:13.359743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:13.360825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 105/216] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:13.366196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.380431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 112/217] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:13.437244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b14d-d61c-449d-abf5-7facd252bbd3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.445086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:13.483371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c688f34-3364-45d7-b2ca-5c3828aaccec None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.518181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.518795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.519074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.519453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.520009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:13.520380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:13.521802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:13.531581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 106/218] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:13.532444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.554448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436b14d-d61c-449d-abf5-7facd252bbd3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.590543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.591187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.591469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.591742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.598718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:13.603148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:13.608892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:13.609773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 113/219] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:13.612937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.656810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d3793-3484-4f1c-8caa-2a09e398be09 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.683636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:13.692106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.693103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.693397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.693671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.694827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:13.696318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:13.697970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 107/220] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:13.700345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.715492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc0739-cfd0-4a59-8a35-a012ccad2a88 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.749986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.750619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.750901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.751191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] PATH_INFO: `/v3/users/769bb0dadaf4406b8df70e95d8a47b1c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.766120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=769bb0dadaf4406b8df70e95d8a47b1c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:13.767578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:13.769315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 114/221] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/users/769bb0dadaf4406b8df70e95d8a47b1c => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:13.770167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.797567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516da2e-3000-404d-b37c-8939c20405a1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.830141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:13.832517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927594c-c732-433a-aa91-6476756814bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.833250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927594c-c732-433a-aa91-6476756814bb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.833459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927594c-c732-433a-aa91-6476756814bb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.833765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927594c-c732-433a-aa91-6476756814bb None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.834413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927594c-c732-433a-aa91-6476756814bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:13.834716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927594c-c732-433a-aa91-6476756814bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:13.836120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927594c-c732-433a-aa91-6476756814bb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:13.844387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 108/222] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:13.845002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927594c-c732-433a-aa91-6476756814bb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.875358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824445de-0bda-4b5a-ad6d-27e3765b0fd7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.903873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:13.910520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.911187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.911472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.911740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.921777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:13.926703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:13.927710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:13.928754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 115/223] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:13.930543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:13.946513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927594c-c732-433a-aa91-6476756814bb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:13.980172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:13.981096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:13.981374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:13.981646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:13.982289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:13.983738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.005672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 109/224] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:13 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:14.006500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.034493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c4279-0486-48d0-8722-e004e1897863 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.052987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:14.078306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.078973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.079212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.079488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.127388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6dcd1-96dc-4349-9d70-e22b68ace37e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.141990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:14.158401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.159885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.168618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.169390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.169652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.169922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] PATH_INFO: `/v3/users/345731bf46dd414d8501dd898ceaf6a3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.171141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] RBAC: Authorizing `identity:get_user(user_id=345731bf46dd414d8501dd898ceaf6a3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.172726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.174628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 110/225] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:14 2022] GET /identity/v3/users/345731bf46dd414d8501dd898ceaf6a3 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:14.175436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.279434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 116/226] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:14.280350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.292788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ee44e-ba60-4533-84ba-ed0b07a4cd47 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.305317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:14.336616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.337370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.337654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.337930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.338762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:14.339433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.340864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.349850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 111/227] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:14.350683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.408710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb61fff-d764-47aa-8f8f-3423af45c0dd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.409059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:14.444761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.445406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.445681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.445953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.447083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.448567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.450229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 117/228] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:14 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:14.450986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.471583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e269066-8546-4ff5-8eb1-fa66aec53234 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.510070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.510717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.511000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.511281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.512636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.514250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.516435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 112/229] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:14 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:14.516970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.548610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:14.555320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c42e57-3039-4f27-9607-45ed5aa4ecb2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.591601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.592394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.592673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.592944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] PATH_INFO: `/v3/users/345731bf46dd414d8501dd898ceaf6a3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.594076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] RBAC: Authorizing `identity:get_user(user_id=345731bf46dd414d8501dd898ceaf6a3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.595619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.597721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 118/230] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:14 2022] GET /identity/v3/users/345731bf46dd414d8501dd898ceaf6a3 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:14.598548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.633137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d05b33-abd6-4ef0-82d0-c6da9747e3df None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.646755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:14.669781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.670457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.670794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.671122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/345731bf46dd414d8501dd898ceaf6a3/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.673297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f1cf0869430492bb74c0f643983bd5b, user_id=345731bf46dd414d8501dd898ceaf6a3, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.674845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.692596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 113/231] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:14 2022] PUT /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/345731bf46dd414d8501dd898ceaf6a3/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:14.693584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.704221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60b4e8-65f1-4baf-b43e-3362f824f5d4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.748590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.749292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.749575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.749846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.752948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=769bb0dadaf4406b8df70e95d8a47b1c) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:14.753249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.757378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.771862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 119/232] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:14 2022] GET /identity/v3/role_assignments?user.id=769bb0dadaf4406b8df70e95d8a47b1c&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:14.854343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eddb8d-2830-4497-a1d3-687173fdf847 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:14.894833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a71b37-47bf-495e-bc15-8216a94a84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:14.895626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a71b37-47bf-495e-bc15-8216a94a84a4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:14.895912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a71b37-47bf-495e-bc15-8216a94a84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:14.896201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a71b37-47bf-495e-bc15-8216a94a84a4 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:14.897502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a71b37-47bf-495e-bc15-8216a94a84a4 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:14.899020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a71b37-47bf-495e-bc15-8216a94a84a4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:14.901130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 114/233] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:14 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:14.907369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:14.940643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:15.028959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3dc46-1e77-4cba-a38d-c880bbba7ede None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:15.075256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450fe31-7ed2-4b73-bade-9261c66c3dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:15.075923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450fe31-7ed2-4b73-bade-9261c66c3dc4 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:15.076219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450fe31-7ed2-4b73-bade-9261c66c3dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:15.076493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450fe31-7ed2-4b73-bade-9261c66c3dc4 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/345731bf46dd414d8501dd898ceaf6a3/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:15.078635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450fe31-7ed2-4b73-bade-9261c66c3dc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f1cf0869430492bb74c0f643983bd5b, user_id=345731bf46dd414d8501dd898ceaf6a3, role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:15.080077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450fe31-7ed2-4b73-bade-9261c66c3dc4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:15.106235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 120/234] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:14 2022] PUT /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/345731bf46dd414d8501dd898ceaf6a3/roles/18b82b06d87e4c93bef42bca94fc658d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:16.334022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1486989-efe1-4695-a483-392542c9fb03 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:16.334756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1486989-efe1-4695-a483-392542c9fb03 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:16.334756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1486989-efe1-4695-a483-392542c9fb03 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:16.336119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 115/235] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:16.341618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd0718-d175-4d61-95f5-8a0a62694cbb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:16.342419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd0718-d175-4d61-95f5-8a0a62694cbb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:16.342894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd0718-d175-4d61-95f5-8a0a62694cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:16.362236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-26dd0718-d175-4d61-95f5-8a0a62694cbb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:16.363764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-26dd0718-d175-4d61-95f5-8a0a62694cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:16.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5432ad4-eaaf-44a9-9bf0-da835fb05148 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:16.438777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5432ad4-eaaf-44a9-9bf0-da835fb05148 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:16.439249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5432ad4-eaaf-44a9-9bf0-da835fb05148 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:16.440865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 116/236] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:16.452379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee252f-babd-45da-85d5-d6c36bfe0dfc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:16.452848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee252f-babd-45da-85d5-d6c36bfe0dfc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:16.453264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee252f-babd-45da-85d5-d6c36bfe0dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:16.471784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 121/237] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:16.485212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a8ee252f-babd-45da-85d5-d6c36bfe0dfc None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:16.486660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a8ee252f-babd-45da-85d5-d6c36bfe0dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:16.536750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:16.578429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 117/238] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:16.924868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd0718-d175-4d61-95f5-8a0a62694cbb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:16.933940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:16.944175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a646d-959d-4851-bfc1-ef39d1cda814 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:16.944175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a646d-959d-4851-bfc1-ef39d1cda814 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:16.944175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a646d-959d-4851-bfc1-ef39d1cda814 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:16.944515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 118/239] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:16.949401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838dff-898b-46c6-91eb-2ae1973376a0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:16.949626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838dff-898b-46c6-91eb-2ae1973376a0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:16.949898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838dff-898b-46c6-91eb-2ae1973376a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:16.975258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-55838dff-898b-46c6-91eb-2ae1973376a0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:16.976470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-55838dff-898b-46c6-91eb-2ae1973376a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:17.055486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd0718-d175-4d61-95f5-8a0a62694cbb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:17.056536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 119/240] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:17.057408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838dff-898b-46c6-91eb-2ae1973376a0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:17.096034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:17.097246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.097525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.097798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.120202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:17.125377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:17.126258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:17.127028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 122/241] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:17.133952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:17.212032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:17.215870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838dff-898b-46c6-91eb-2ae1973376a0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:17.218346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:17.257859 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194de28-c5da-4845-bb2e-c0b7bd3f7c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:17.258674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194de28-c5da-4845-bb2e-c0b7bd3f7c57 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.259052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194de28-c5da-4845-bb2e-c0b7bd3f7c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.259423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194de28-c5da-4845-bb2e-c0b7bd3f7c57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.260851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194de28-c5da-4845-bb2e-c0b7bd3f7c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:17.262410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194de28-c5da-4845-bb2e-c0b7bd3f7c57 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:17.263835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d6a6a-4d62-4216-8a51-30a6f83c3286 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:17.264619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 120/242] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:17.266009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d63e-7ca5-40ac-9c9f-81cbe798dae1 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.266387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d63e-7ca5-40ac-9c9f-81cbe798dae1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.266757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1d63e-7ca5-40ac-9c9f-81cbe798dae1 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.268255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 121/243] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:17.273455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d63e-7ca5-40ac-9c9f-81cbe798dae1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:17.306989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caaca77-fd7e-46c9-9035-ccdd6fa8a454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:17.307607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaca77-fd7e-46c9-9035-ccdd6fa8a454 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.308316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaca77-fd7e-46c9-9035-ccdd6fa8a454 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.308642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caaca77-fd7e-46c9-9035-ccdd6fa8a454 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.309337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaca77-fd7e-46c9-9035-ccdd6fa8a454 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:17.309647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaca77-fd7e-46c9-9035-ccdd6fa8a454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:17.311082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caaca77-fd7e-46c9-9035-ccdd6fa8a454 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:17.324919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 123/244] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:17.324919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd121d-db59-4075-921d-f47b2a039eb3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.324919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd121d-db59-4075-921d-f47b2a039eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.324919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd121d-db59-4075-921d-f47b2a039eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.341479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9abd121d-db59-4075-921d-f47b2a039eb3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:17.342822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9abd121d-db59-4075-921d-f47b2a039eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:17.404405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1d63e-7ca5-40ac-9c9f-81cbe798dae1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:17.436172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 124/245] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:17.436172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd121d-db59-4075-921d-f47b2a039eb3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:17.439457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:17.440121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.440423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.440735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.441389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:17.443092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:17.469693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 122/246] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:17 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:17.470570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:17.533271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:17.543322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:17.614856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd121d-db59-4075-921d-f47b2a039eb3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:17.654909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a13f0-2000-4ee5-9492-9328026d199e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:17.657127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4926ad-ca6a-424d-99dd-db6be1036d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:17.657937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4926ad-ca6a-424d-99dd-db6be1036d5f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.658365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4926ad-ca6a-424d-99dd-db6be1036d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.658762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4926ad-ca6a-424d-99dd-db6be1036d5f None admin] PATH_INFO: `/v3/users/b02a4eb393d849419bb6105e9f5c9836` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.673746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4926ad-ca6a-424d-99dd-db6be1036d5f None admin] RBAC: Authorizing `identity:get_user(user_id=b02a4eb393d849419bb6105e9f5c9836)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:17.675219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4926ad-ca6a-424d-99dd-db6be1036d5f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:17.677109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 125/247] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:17 2022] GET /identity/v3/users/b02a4eb393d849419bb6105e9f5c9836 => generated 297 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:17.678275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39c858-9824-4f1a-ba8b-b1463560b87b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.684997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39c858-9824-4f1a-ba8b-b1463560b87b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.684997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39c858-9824-4f1a-ba8b-b1463560b87b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.684997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 126/248] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:17.684997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39c858-9824-4f1a-ba8b-b1463560b87b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:17.706466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78476fcb-ac88-44f8-9035-c191dd211a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:17.707923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78476fcb-ac88-44f8-9035-c191dd211a7c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.708607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78476fcb-ac88-44f8-9035-c191dd211a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.709021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78476fcb-ac88-44f8-9035-c191dd211a7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.728830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78476fcb-ac88-44f8-9035-c191dd211a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-78476fcb-ac88-44f8-9035-c191dd211a7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:17.734481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:17.735331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:17.736362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 123/249] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:17.737182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5f94-e250-486b-9d3a-c70d574a2788 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.737182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5f94-e250-486b-9d3a-c70d574a2788 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.737384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5f94-e250-486b-9d3a-c70d574a2788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.757763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c99b5f94-e250-486b-9d3a-c70d574a2788 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:17.759092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c99b5f94-e250-486b-9d3a-c70d574a2788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:17.810535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39c858-9824-4f1a-ba8b-b1463560b87b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:17.829284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:17.839162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 124/250] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:17.840030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5f94-e250-486b-9d3a-c70d574a2788 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:17.853123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:17.853800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:17.854163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:17.854440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:17.865672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:17.954942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:17.957032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.010536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5f94-e250-486b-9d3a-c70d574a2788 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.076380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 127/251] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:18.077898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.098035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:18.099180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.099846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.100478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.100478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.123202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:18.128138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:18.129053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:18.130315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 125/252] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:18.130951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.203396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651378a-d6ce-4bf1-83a5-a0f0f27a75ab None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.207328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:18.244977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.245181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625f8f72-7552-4a35-941e-2476062de57b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.245603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.245882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.246153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.246941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:18.247359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.248902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.262807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 128/253] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.263637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.286090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.287057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.288029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.288545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.290341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.292705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.295359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 126/254] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.301032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.354610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:18.378096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171b944-5431-463b-950d-b972b6c0ac20 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.414291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.415026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.415504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.415918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.427504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:18.427648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78a2b9-2b6e-448a-9c27-d1b8afd6de3f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:18.433737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:18.434748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:18.435528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:18.442447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 129/255] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:18.443505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.463464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.464064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.464425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.464715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.465478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:18.465786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.467194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.476231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 127/256] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.481975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.485847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:18.555676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c709fe08-ce5b-411f-b513-55103711542a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.588580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfdd8d9-4080-4dd0-ab68-c1e4a6074cc3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.592167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.593185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.593808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.594415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] PATH_INFO: `/v3/users/345731bf46dd414d8501dd898ceaf6a3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.596417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] RBAC: Authorizing `identity:get_user(user_id=345731bf46dd414d8501dd898ceaf6a3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.598149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.600027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 130/257] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/users/345731bf46dd414d8501dd898ceaf6a3 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.605171 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.617358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:18.626003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.626797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.627340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.627770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.628763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] RBAC: Adding query filter params (role.id=e9db72a727fb4cd1a83a7cd4965c51a3, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=b02a4eb393d849419bb6105e9f5c9836) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:18.629215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.630907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.637394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 128/258] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/role_assignments?user.id=b02a4eb393d849419bb6105e9f5c9836&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=e9db72a727fb4cd1a83a7cd4965c51a3 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.638589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.713408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:18.724546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3060e62-a634-428d-93f9-cbfee2ad5565 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.753378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50283391-115d-4e71-9f72-edf4c9617a20 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.763963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.764897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.765461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.765931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.766792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:18.767310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.769109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.778696 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 131/259] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.779979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.792024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.792722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.793003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.793281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] PATH_INFO: `/v3/users/769bb0dadaf4406b8df70e95d8a47b1c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.794450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] RBAC: Authorizing `identity:get_user(user_id=769bb0dadaf4406b8df70e95d8a47b1c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.795943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.797850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 129/260] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/users/769bb0dadaf4406b8df70e95d8a47b1c => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.803630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.862060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:18.901415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee480c5-ba20-41d4-b67b-951efe59ac6a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.919463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbe143-e650-4d45-9a96-4bb65f986d9e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:18.939216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.939889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.940309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.940752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.957659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:18.958733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:18.959274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:18.959729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] PATH_INFO: `/v3/users/345731bf46dd414d8501dd898ceaf6a3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:18.961090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] RBAC: Authorizing `identity:get_user(user_id=345731bf46dd414d8501dd898ceaf6a3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:18.962931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:18.965120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 130/261] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/users/345731bf46dd414d8501dd898ceaf6a3 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:18.966174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:18.976305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:19.024616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:19.026457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:19.078685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624767b-2985-4cb9-8bfa-c006af4e0a64 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:19.101760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:19.119021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:19.120115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.120759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.121230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.122583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:19.124515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:19.126382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 131/262] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:19.131556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:19.144126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 132/263] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:19.145437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:19.209069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:19.248296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa9598-ec68-4333-bbc3-558e080e6bae None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:19.278699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:19.279470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.279829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a456fb-8671-484b-a53a-9ba18c7a6054 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:19.279956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.283716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.345476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:19.346449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.346903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.347318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/769bb0dadaf4406b8df70e95d8a47b1c/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.351142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f1cf0869430492bb74c0f643983bd5b, user_id=769bb0dadaf4406b8df70e95d8a47b1c, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:19.352657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:19.353353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:19.354351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:19.389973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 133/264] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:19 2022] PUT /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/769bb0dadaf4406b8df70e95d8a47b1c/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:19.391375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:19.461056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:19.482854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:19.547769 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 132/265] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:19.555932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:19.584872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd1fbc-2a63-45f1-8d7e-01ee3c5cfe50 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:19.624204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:19.625141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.625593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.631970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.634880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:19.637086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:19.639022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 134/266] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:19 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:19.645416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:19.746607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c11f9-5426-4dd9-a567-63df2472e3aa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:19.784536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb9e7b-f7c6-4079-b356-291a243479d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:19.785240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb9e7b-f7c6-4079-b356-291a243479d8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.785484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb9e7b-f7c6-4079-b356-291a243479d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.785783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb9e7b-f7c6-4079-b356-291a243479d8 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.786948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb9e7b-f7c6-4079-b356-291a243479d8 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:19.788533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb9e7b-f7c6-4079-b356-291a243479d8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:19.790197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 133/267] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:19 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:19.793594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf66b1-aaeb-42ff-b9fb-83ce6bf05235 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.793879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf66b1-aaeb-42ff-b9fb-83ce6bf05235 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.794154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf66b1-aaeb-42ff-b9fb-83ce6bf05235 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.795539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 134/268] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:19.800927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0661e-d7ce-4896-8638-7d84c03150c6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.801442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0661e-d7ce-4896-8638-7d84c03150c6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.801726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0661e-d7ce-4896-8638-7d84c03150c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.821198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5cd0661e-d7ce-4896-8638-7d84c03150c6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:19.821833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:19.822405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5cd0661e-d7ce-4896-8638-7d84c03150c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:19.824763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520bc60-ad2a-4884-9ab8-0b130220ee2a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:19.863363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:19.864068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:19.864597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:19.865056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:19.865857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=345731bf46dd414d8501dd898ceaf6a3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:19.866227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:19.867800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:19.874514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 135/269] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:19 2022] GET /identity/v3/role_assignments?user.id=345731bf46dd414d8501dd898ceaf6a3&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:19.874913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:19.875350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:19.908490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 135/270] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:20.000032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b572b1-b127-4993-8d77-6b3ed70dc24e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:20.012604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55794-03cd-4ddc-abda-e9ab78fa9d1c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.012885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55794-03cd-4ddc-abda-e9ab78fa9d1c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.013156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb55794-03cd-4ddc-abda-e9ab78fa9d1c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.014763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 136/271] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:20.020010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b86d3-2e19-4622-87da-2263f119a4b0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.020334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b86d3-2e19-4622-87da-2263f119a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.020614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b86d3-2e19-4622-87da-2263f119a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.037573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:20.038189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.038462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.038729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.039522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] RBAC: Adding query filter params (role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=345731bf46dd414d8501dd898ceaf6a3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:20.039832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:20.041313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:20.042207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-df8b86d3-2e19-4622-87da-2263f119a4b0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:20.043650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-df8b86d3-2e19-4622-87da-2263f119a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:20.047746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 136/272] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:19 2022] GET /identity/v3/role_assignments?user.id=345731bf46dd414d8501dd898ceaf6a3&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:20.166513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 137/273] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:20.350230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:20.380351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:20.461833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b86d3-2e19-4622-87da-2263f119a4b0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:20.476925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ff95f-b05d-404e-b247-b4323c2acffa None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:20.521919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:20.522571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.522946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.523230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.538260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:20.544037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:20.545176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:20.546149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 137/274] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:20.553839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:20.590709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:20.594522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b86d3-2e19-4622-87da-2263f119a4b0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:20.634325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:20.635440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.635917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.636414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.637979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:20.639720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:20.641950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 138/275] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:20.649553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:20.661002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:20.679353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b066e9-4860-4643-a869-54a06d557fb8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:20.714051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:20.715098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.715608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.716065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.716945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:20.717523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:20.719291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:20.728277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 138/276] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:20.734860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:20.769980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fb6d6-55bc-4db0-8319-406d2271ad48 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:20.806729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:20.807709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.808204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.808792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.809750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:20.811473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:20.836641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 139/277] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:20 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:48:20.859917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a57ea-8644-432d-8f27-ea33a68f9138 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:20.900283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd269c0-5c4b-4c8d-b5af-650129bea29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:20.901383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd269c0-5c4b-4c8d-b5af-650129bea29a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:20.901855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd269c0-5c4b-4c8d-b5af-650129bea29a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:20.902345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd269c0-5c4b-4c8d-b5af-650129bea29a None admin] PATH_INFO: `/v3/users/5d582186fa274ee3bf544d04ba305619` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:20.918953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd269c0-5c4b-4c8d-b5af-650129bea29a None admin] RBAC: Authorizing `identity:get_user(user_id=5d582186fa274ee3bf544d04ba305619)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:20.920920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd269c0-5c4b-4c8d-b5af-650129bea29a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:20.923198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 139/278] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:20 2022] GET /identity/v3/users/5d582186fa274ee3bf544d04ba305619 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:20.933349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:20.962792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:21.043550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1ac54-3c6c-4590-a4fa-91635674465d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:21.078425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235c3c1-27c8-49a3-a3d3-22f386cfdc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:21.079036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235c3c1-27c8-49a3-a3d3-22f386cfdc76 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:21.079314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235c3c1-27c8-49a3-a3d3-22f386cfdc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:21.079584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235c3c1-27c8-49a3-a3d3-22f386cfdc76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:21.080345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235c3c1-27c8-49a3-a3d3-22f386cfdc76 None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, user.id=5d582186fa274ee3bf544d04ba305619, scope.system=all) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:21.080655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235c3c1-27c8-49a3-a3d3-22f386cfdc76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:21.082321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235c3c1-27c8-49a3-a3d3-22f386cfdc76 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:21.087998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 140/279] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Sep 5 16:48:20 2022] GET /identity/v3/role_assignments?user.id=5d582186fa274ee3bf544d04ba305619&scope.system=all&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:21.383503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf1c7-6556-47de-856d-8a772186db3c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:21.383791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf1c7-6556-47de-856d-8a772186db3c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:21.384063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20faf1c7-6556-47de-856d-8a772186db3c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:21.385766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 140/280] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:21.392240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d1a6-9515-4e5e-b293-1376e3e36c41 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:21.392240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d1a6-9515-4e5e-b293-1376e3e36c41 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:21.392240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987d1a6-9515-4e5e-b293-1376e3e36c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:21.421610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5987d1a6-9515-4e5e-b293-1376e3e36c41 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:21.423121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5987d1a6-9515-4e5e-b293-1376e3e36c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:21.518442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 141/281] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:21.800946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20faf1c7-6556-47de-856d-8a772186db3c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:21.805899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:21.937761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20faf1c7-6556-47de-856d-8a772186db3c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:21.986179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:21.986789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07717f-954f-43da-86e7-d2a994b00796 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:21.987249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:21.987249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:21.987377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07717f-954f-43da-86e7-d2a994b00796 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:21.987450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:21.987515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07717f-954f-43da-86e7-d2a994b00796 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:21.990361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 142/282] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:21.994564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0b24-9f9c-4dfe-996c-64a7fcc69776 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:21.994850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0b24-9f9c-4dfe-996c-64a7fcc69776 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:21.995124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c0b24-9f9c-4dfe-996c-64a7fcc69776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:22.005192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:22.010099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:22.010945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:22.011719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 141/283] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:22.013930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-204c0b24-9f9c-4dfe-996c-64a7fcc69776 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:22.015220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-204c0b24-9f9c-4dfe-996c-64a7fcc69776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:22.017037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.038837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:22.088231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:22.157427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 143/284] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:22.159731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd0744-2f86-4bc4-87c2-6110b61409cf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:22.200277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:22.200898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:22.201178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:22.201450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:22.202179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:22.202486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:22.203992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:22.223294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 142/285] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:22.228973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0b24-9f9c-4dfe-996c-64a7fcc69776 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.386040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c0b24-9f9c-4dfe-996c-64a7fcc69776 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:22.433648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:22.434282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:22.434558 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:22.434826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] PATH_INFO: `/v3/users/b02a4eb393d849419bb6105e9f5c9836` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:22.440801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] RBAC: Authorizing `identity:get_user(user_id=b02a4eb393d849419bb6105e9f5c9836)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:22.442351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:22.444266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 144/286] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:22 2022] GET /identity/v3/users/b02a4eb393d849419bb6105e9f5c9836 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:22.449841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.474000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.493227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:22.579331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b922c98-e43a-4770-9758-21862359e81d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:22.615027 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:22.615686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:22.616059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:22.616418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:22.617984 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737505d-bf79-4ef0-b33d-8a91f3cb12f1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:22.632250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:22.633046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:22.633893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 145/287] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:22.645362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.661941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:22.683496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:22.684921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:22.684921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:22.684921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:22.688858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:22.690387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:22.696181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 143/288] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:22 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:22.706917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.740587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:22.759230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7c229-98bb-4320-9f00-de3170115907 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:22.797678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:22.798646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:22.799114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:22.799570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:22.800447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:22.800967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:22.802677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:22.812049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 146/289] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:22.817540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.914519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792007c1-a27b-4ccd-b9fc-54c1e2361d35 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:22.941846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fc32c-5831-4293-a85f-526f1b2d419a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:22.977680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:22.978665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:22.979187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:22.979699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] PATH_INFO: `/v3/users/769bb0dadaf4406b8df70e95d8a47b1c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:22.981539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] RBAC: Authorizing `identity:get_user(user_id=769bb0dadaf4406b8df70e95d8a47b1c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:22.983213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:22.985346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 147/290] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:22 2022] GET /identity/v3/users/769bb0dadaf4406b8df70e95d8a47b1c => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:22.990811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:22.995852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e7226-6438-4f84-b5d2-bbb27c320377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:22.996114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e7226-6438-4f84-b5d2-bbb27c320377 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:22.996388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e7226-6438-4f84-b5d2-bbb27c320377 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:22.996659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e7226-6438-4f84-b5d2-bbb27c320377 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/b02a4eb393d849419bb6105e9f5c9836/roles/e9db72a727fb4cd1a83a7cd4965c51a3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.001415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e7226-6438-4f84-b5d2-bbb27c320377 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f1cf0869430492bb74c0f643983bd5b, user_id=b02a4eb393d849419bb6105e9f5c9836, role_id=e9db72a727fb4cd1a83a7cd4965c51a3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:23.002858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e7226-6438-4f84-b5d2-bbb27c320377 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:23.016569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:23.050643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 144/291] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:48:22 2022] PUT /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b/users/b02a4eb393d849419bb6105e9f5c9836/roles/e9db72a727fb4cd1a83a7cd4965c51a3 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:48:23.127538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634351c-1374-4a17-b3c7-7d60489dce02 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:23.185169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:23.186163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.186629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.187085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.264090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:23.266007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:23.275447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:23.360358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af3e37-ede8-451a-834a-4447c2b83f54 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.360641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af3e37-ede8-451a-834a-4447c2b83f54 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.360915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20af3e37-ede8-451a-834a-4447c2b83f54 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.362267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 145/292] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:23.367780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a795ef-70fd-4270-8629-03e81e2b1459 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.368082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a795ef-70fd-4270-8629-03e81e2b1459 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.368377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a795ef-70fd-4270-8629-03e81e2b1459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.397648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-44a795ef-70fd-4270-8629-03e81e2b1459 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:23.398854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-44a795ef-70fd-4270-8629-03e81e2b1459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:23.401729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 148/293] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:23.406879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:23.455342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:23.490261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 146/294] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:23.503557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f727be-0078-48a1-8c20-3660dfa27087 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.504327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f727be-0078-48a1-8c20-3660dfa27087 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.504860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f727be-0078-48a1-8c20-3660dfa27087 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.508012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 147/295] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:23.512759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de45b-4679-416a-a865-d8d59cec4e4d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.513271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de45b-4679-416a-a865-d8d59cec4e4d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.513761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de45b-4679-416a-a865-d8d59cec4e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.533162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-750de45b-4679-416a-a865-d8d59cec4e4d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:23.534855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-750de45b-4679-416a-a865-d8d59cec4e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:23.573350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1814d-839e-4357-aa62-dfd1752dc740 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:23.586799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:23.620251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 148/296] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:23.636056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875ac24-4db0-4f73-9865-883c528eff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:23.636740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ac24-4db0-4f73-9865-883c528eff42 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.637023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ac24-4db0-4f73-9865-883c528eff42 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.637305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875ac24-4db0-4f73-9865-883c528eff42 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.638439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875ac24-4db0-4f73-9865-883c528eff42 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:23.639893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f875ac24-4db0-4f73-9865-883c528eff42 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:23.641942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 149/297] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:23 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:23.647481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750de45b-4679-416a-a865-d8d59cec4e4d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:23.654038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa58c48-ec91-42cd-b49b-23a20b2bb7d6 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.654322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa58c48-ec91-42cd-b49b-23a20b2bb7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.654596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa58c48-ec91-42cd-b49b-23a20b2bb7d6 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.656069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 150/298] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:23.661039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d1e8-1ccc-4320-a219-24b5435354e9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.661321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d1e8-1ccc-4320-a219-24b5435354e9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.661594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d1e8-1ccc-4320-a219-24b5435354e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.680886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cfc2d1e8-1ccc-4320-a219-24b5435354e9 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:23.682330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cfc2d1e8-1ccc-4320-a219-24b5435354e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:23.774408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 151/299] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:23.807311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750de45b-4679-416a-a865-d8d59cec4e4d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:23.822899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:23.851754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:23.852411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:23.852692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:23.852966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:23.853706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=769bb0dadaf4406b8df70e95d8a47b1c) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:23.854014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:23.855792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:23.869222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 149/300] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:23 2022] GET /identity/v3/role_assignments?user.id=769bb0dadaf4406b8df70e95d8a47b1c&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:23.879085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2d1e8-1ccc-4320-a219-24b5435354e9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:23.921197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.080968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2d1e8-1ccc-4320-a219-24b5435354e9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.083826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:24.091636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcb8ce-1f29-4e3a-912f-235725237d0d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.123733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.124520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.125025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.125306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.137486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.137865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:24.138126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.138406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.138678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:24.143269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:24.144123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:24.144992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 152/301] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:24.146100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.156112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:24.158729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:24.159671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:24.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 150/302] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:24.161576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.214572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:24.272115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7a063-940b-4d34-8137-9d1088ac3307 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.275899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ae3a6-862a-4d23-9f76-106d9dc75038 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.307939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.308610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.308914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.309082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.309160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.309516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.309763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.310028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.310681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:24.310875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.310980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.312356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.312788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.314773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 153/303] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:24.315726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.328612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 151/304] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:24.328612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.352981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:24.436424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54345e-1f5e-4216-bc83-4078fb7d640c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.440219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:24.443222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9db8f-a5e3-42f2-a73d-35a11338455f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.472644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.473288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.473556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.473832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.474552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:24.474854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.476299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.480280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.481238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.481517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.481791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] PATH_INFO: `/v3/groups` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.482568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.484047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.486074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 154/305] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:24.486933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.506112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 152/306] 10.222.0.23 () {60 vars in 1134 bytes} [Mon Sep 5 16:48:24 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:24.506947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.628280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:24.641542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c5cfb-244e-4143-8bb2-6d312719ab5f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.651024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262aba8-063d-4eb0-a4ac-6fac3b82730e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.680530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.681130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.681415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.681692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] PATH_INFO: `/v3/users/0fad2b0026e84e08b666772ac0cd58f3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.694641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.695280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.695556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.695826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] PATH_INFO: `/v3/users/5d582186fa274ee3bf544d04ba305619` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.696964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2b0026e84e08b666772ac0cd58f3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.698301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] RBAC: Authorizing `identity:get_user(user_id=5d582186fa274ee3bf544d04ba305619)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.698722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.699729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.700757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 155/307] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:24 2022] GET /identity/v3/users/0fad2b0026e84e08b666772ac0cd58f3 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:24.706559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 153/308] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:24 2022] GET /identity/v3/users/5d582186fa274ee3bf544d04ba305619 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:24.707510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.709392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:24.747338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:24.830912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43285a9-7745-4f24-8b3b-d07633daaac0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.839932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d463147-2612-4034-a9f6-c328b44aa3af None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:24.869598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747710f-5d49-41d8-8816-41576a9fe2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.870226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747710f-5d49-41d8-8816-41576a9fe2dc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.870507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747710f-5d49-41d8-8816-41576a9fe2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.870777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747710f-5d49-41d8-8816-41576a9fe2dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.871604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747710f-5d49-41d8-8816-41576a9fe2dc None admin] RBAC: Adding query filter params (role.id=e9db72a727fb4cd1a83a7cd4965c51a3, user.id=0fad2b0026e84e08b666772ac0cd58f3, scope.system=all) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:24.871919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747710f-5d49-41d8-8816-41576a9fe2dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.873579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747710f-5d49-41d8-8816-41576a9fe2dc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.880915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 154/309] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Sep 5 16:48:24 2022] GET /identity/v3/role_assignments?user.id=0fad2b0026e84e08b666772ac0cd58f3&scope.system=all&role.id=e9db72a727fb4cd1a83a7cd4965c51a3 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:24.886770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c18166-3e96-4a9e-b758-4fc7d479847c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:24.887478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18166-3e96-4a9e-b758-4fc7d479847c None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:24.887758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18166-3e96-4a9e-b758-4fc7d479847c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:24.888031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c18166-3e96-4a9e-b758-4fc7d479847c None admin] PATH_INFO: `/v3/system/users/5d582186fa274ee3bf544d04ba305619/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:24.897734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c18166-3e96-4a9e-b758-4fc7d479847c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d582186fa274ee3bf544d04ba305619, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:24.897734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c18166-3e96-4a9e-b758-4fc7d479847c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:24.908969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 156/310] 10.222.0.23 () {58 vars in 1334 bytes} [Mon Sep 5 16:48:24 2022] PUT /identity/v3/system/users/5d582186fa274ee3bf544d04ba305619/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:25.386954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0da5f-ad62-4ade-be43-1f35cf2be5ac None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:25.387465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0da5f-ad62-4ade-be43-1f35cf2be5ac None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:25.387465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0da5f-ad62-4ade-be43-1f35cf2be5ac None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:25.389107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 155/311] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:25.394721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2786b3-c722-4bb0-a638-fb6707cc0552 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:25.395210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2786b3-c722-4bb0-a638-fb6707cc0552 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:25.395414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2786b3-c722-4bb0-a638-fb6707cc0552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:25.421589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0e2786b3-c722-4bb0-a638-fb6707cc0552 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:25.422795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0e2786b3-c722-4bb0-a638-fb6707cc0552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:25.435706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:25.517652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 157/312] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:25.813826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0da5f-ad62-4ade-be43-1f35cf2be5ac None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:25.914566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:25.924973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0da5f-ad62-4ade-be43-1f35cf2be5ac None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:25.963682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:25.964753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:25.965222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:25.965677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:25.982504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:25.987740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:25.988901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:25.989713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:25.992031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 156/313] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:25.997987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2786b3-c722-4bb0-a638-fb6707cc0552 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:26.104789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2786b3-c722-4bb0-a638-fb6707cc0552 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:26.143983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:26.144630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.145063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.145377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.146039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:26.146340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:26.147735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:26.156705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 158/314] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:26.165592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:26.269381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415f2bc-3825-4e2b-b5ed-a2db1f7e653a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:26.303964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:26.305031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.305495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.305974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] PATH_INFO: `/v3/users/b02a4eb393d849419bb6105e9f5c9836` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.307293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] RBAC: Authorizing `identity:get_user(user_id=b02a4eb393d849419bb6105e9f5c9836)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:26.308957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:26.311074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 157/315] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:26 2022] GET /identity/v3/users/b02a4eb393d849419bb6105e9f5c9836 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:26.317406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:26.331602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:26.422163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4c57e-361a-4946-8c76-d482025670e7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:26.456421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ff0e3-e0f2-4d89-bcbf-e46a921a22b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:26.457036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ff0e3-e0f2-4d89-bcbf-e46a921a22b8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.457367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ff0e3-e0f2-4d89-bcbf-e46a921a22b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.457640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ff0e3-e0f2-4d89-bcbf-e46a921a22b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.525930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ff0e3-e0f2-4d89-bcbf-e46a921a22b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:26.527502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ff0e3-e0f2-4d89-bcbf-e46a921a22b8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:26.537708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:26.628642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 159/316] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:26.634094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:26.661058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:26.676822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef560d-5c2e-4c97-9f3e-d8afe076dd98 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.677170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef560d-5c2e-4c97-9f3e-d8afe076dd98 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.677362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef560d-5c2e-4c97-9f3e-d8afe076dd98 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.678870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 160/317] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:26.684831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a851d5f-28e3-4e9d-b6d6-7c0abb4ed88c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.685305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a851d5f-28e3-4e9d-b6d6-7c0abb4ed88c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.685788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a851d5f-28e3-4e9d-b6d6-7c0abb4ed88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.708677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0a851d5f-28e3-4e9d-b6d6-7c0abb4ed88c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:26.711203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a851d5f-28e3-4e9d-b6d6-7c0abb4ed88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:26.741645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc1316-c5f6-47f5-9582-9c68812b8471 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:26.777247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:26.778216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.778682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.779122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.780425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:26.782065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:26.788816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 158/318] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:26 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:26.789889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:26.797039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 161/319] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:26.884115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:26.895162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111564c-46c8-41db-ab2c-6b56b28dde9f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.895456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111564c-46c8-41db-ab2c-6b56b28dde9f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.895691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111564c-46c8-41db-ab2c-6b56b28dde9f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.897327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 162/320] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:26.898493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3601991-2381-4c64-be6f-cf2e9b8a5506 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:26.905492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86f0da-9cca-4a0e-a11d-50cf34e927c7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.906084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86f0da-9cca-4a0e-a11d-50cf34e927c7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.906551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86f0da-9cca-4a0e-a11d-50cf34e927c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.925882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4d86f0da-9cca-4a0e-a11d-50cf34e927c7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:26.927506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4d86f0da-9cca-4a0e-a11d-50cf34e927c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:26.932818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:26.933433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:26.933705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:26.933971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:26.934687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] RBAC: Adding query filter params (role.id=e9db72a727fb4cd1a83a7cd4965c51a3, scope.project.id=7f1cf0869430492bb74c0f643983bd5b, user.id=b02a4eb393d849419bb6105e9f5c9836) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:26.934985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:26.936444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:26.943172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 159/321] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:26 2022] GET /identity/v3/role_assignments?user.id=b02a4eb393d849419bb6105e9f5c9836&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=e9db72a727fb4cd1a83a7cd4965c51a3 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:26.964072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:27.015363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 163/322] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:27.069706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.101454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f1ff3-e169-46d2-b280-6d6a9c033082 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.102064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f1ff3-e169-46d2-b280-6d6a9c033082 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.102693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f1ff3-e169-46d2-b280-6d6a9c033082 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.104509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 164/323] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:27.110043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ea228-4c70-45dc-8afa-f93bdd12c9f4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.110291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ea228-4c70-45dc-8afa-f93bdd12c9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.110535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ea228-4c70-45dc-8afa-f93bdd12c9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.131281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fa1ea228-4c70-45dc-8afa-f93bdd12c9f4 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:27.132517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fa1ea228-4c70-45dc-8afa-f93bdd12c9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:27.186968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece5e3d-4988-4a20-88eb-7f1ffcd3ba63 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.212258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 165/324] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:27.231565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:27.232593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.233071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.233537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.234859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:27.236535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:27.238716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 160/325] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:27.244595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ea228-4c70-45dc-8afa-f93bdd12c9f4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.289589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.313059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:27.344391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:27.348780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ea228-4c70-45dc-8afa-f93bdd12c9f4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.388347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:27.388963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.389234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.389598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] PATH_INFO: `/v3/groups` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.390243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:27.391693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:27.409229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 166/326] 10.222.0.23 () {60 vars in 1134 bytes} [Mon Sep 5 16:48:27 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:27.418788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98c428-d1f9-4fec-abba-e23a814c282a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.458750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:27.459701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.460191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.460638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.472748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:27.478586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:27.479552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:27.480792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:27.481387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 161/327] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:27.483930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.487851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.584090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:27.605701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fe44e-ed72-4c0d-ade5-f9efc4d3e11e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.608103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f618dd-3d50-4e2a-aa74-fb644bb74e5f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.649449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:27.650093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.650341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.650614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.651327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:27.651583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:27.653116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:27.653651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:27.654524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.654997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.655461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.664252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 162/328] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:27.670207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:27.670452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:27.675650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:27.676712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:27.677501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:27.678086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 167/329] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:27.683504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.704711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:27.792483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ab57d-2e1f-487b-aa11-c40478c8eece None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.820026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a3533-4f14-4e68-8edb-8d25425624ba None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.846865 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:27.847553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.847962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.850912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] PATH_INFO: `/v3/users/5d582186fa274ee3bf544d04ba305619` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.850912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] RBAC: Authorizing `identity:get_user(user_id=5d582186fa274ee3bf544d04ba305619)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:27.851104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:27.853045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 163/330] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/users/5d582186fa274ee3bf544d04ba305619 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:27.859741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.866086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:27.866712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:27.866989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:27.867268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:27.867931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:27.868280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:27.869647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:27.873576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:27.883242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 168/331] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:27.889883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:27.988978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0393d3c-e812-4ba4-9e60-392ceaafc55c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:27.995030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:28.019218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc64569b-8367-44a0-bba1-9b96e56977df None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:28.033885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:28.034508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:28.034804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:28.035335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:28.036102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] RBAC: Adding query filter params (role.id=e68472fdebed4f58b495dc466408a9e4, user.id=5d582186fa274ee3bf544d04ba305619, scope.system=all) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:28.036441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:28.038157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:28.055452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 164/332] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/role_assignments?user.id=5d582186fa274ee3bf544d04ba305619&scope.system=all&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:28.066156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744d6c3-99d1-4675-bf44-1ced6fa9a2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:28.066965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d6c3-99d1-4675-bf44-1ced6fa9a2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:28.067311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d6c3-99d1-4675-bf44-1ced6fa9a2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:28.067655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744d6c3-99d1-4675-bf44-1ced6fa9a2d1 None admin] PATH_INFO: `/v3/users/0fad2b0026e84e08b666772ac0cd58f3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:28.073252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744d6c3-99d1-4675-bf44-1ced6fa9a2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2b0026e84e08b666772ac0cd58f3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:28.074874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744d6c3-99d1-4675-bf44-1ced6fa9a2d1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:28.077090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 169/333] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:27 2022] GET /identity/v3/users/0fad2b0026e84e08b666772ac0cd58f3 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:28.083830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:28.169636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:28.210431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79894015-4e6d-4ddc-9fc0-e538aed9dfcd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:28.247579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfaaa32-46dd-424a-b271-d8d964f88afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:28.248332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaaa32-46dd-424a-b271-d8d964f88afa None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:28.248615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaaa32-46dd-424a-b271-d8d964f88afa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:28.248889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaaa32-46dd-424a-b271-d8d964f88afa None admin] PATH_INFO: `/v3/system/users/0fad2b0026e84e08b666772ac0cd58f3/roles/e9db72a727fb4cd1a83a7cd4965c51a3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:28.250355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaaa32-46dd-424a-b271-d8d964f88afa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0fad2b0026e84e08b666772ac0cd58f3, role_id=e9db72a727fb4cd1a83a7cd4965c51a3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:28.251733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfaaa32-46dd-424a-b271-d8d964f88afa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:28.265578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 165/334] 10.222.0.23 () {58 vars in 1334 bytes} [Mon Sep 5 16:48:28 2022] PUT /identity/v3/system/users/0fad2b0026e84e08b666772ac0cd58f3/roles/e9db72a727fb4cd1a83a7cd4965c51a3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:48:29.920348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0fa5-c105-4acf-933c-36ade4bc03db None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:29.921346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0fa5-c105-4acf-933c-36ade4bc03db None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:29.921820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d0fa5-c105-4acf-933c-36ade4bc03db None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:29.923364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 170/335] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:29.928927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae15f69-e238-4ee3-8b62-2cd9808d5d9e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:29.929821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae15f69-e238-4ee3-8b62-2cd9808d5d9e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:29.930280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae15f69-e238-4ee3-8b62-2cd9808d5d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:29.957266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6ae15f69-e238-4ee3-8b62-2cd9808d5d9e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:29.958929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6ae15f69-e238-4ee3-8b62-2cd9808d5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:30.040893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 166/336] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:30.083564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818574b-ab0f-493a-a410-0b848780bb0c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:30.084195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818574b-ab0f-493a-a410-0b848780bb0c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:30.084690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818574b-ab0f-493a-a410-0b848780bb0c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:30.086502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 171/337] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:30.091653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53b250-76e2-4e90-9a1c-82d0c1306f0a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:30.091946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53b250-76e2-4e90-9a1c-82d0c1306f0a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:30.092249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53b250-76e2-4e90-9a1c-82d0c1306f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:30.117726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bb53b250-76e2-4e90-9a1c-82d0c1306f0a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:30.118925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bb53b250-76e2-4e90-9a1c-82d0c1306f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:30.190579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:30.201741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 167/338] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:30.459072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818574b-ab0f-493a-a410-0b848780bb0c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:30.570721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818574b-ab0f-493a-a410-0b848780bb0c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:30.609177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:30.610207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:30.610733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:30.611192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:30.613586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53b250-76e2-4e90-9a1c-82d0c1306f0a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:30.626690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:30.632307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:30.633149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:30.633908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:30.636243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 172/339] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:30.642068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:30.655592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:30.766954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a0a9ab-a9df-4083-8a03-5d65f2cabf00 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:30.814024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:30.814229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53b250-76e2-4e90-9a1c-82d0c1306f0a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:30.815187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:30.815653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:30.816105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:30.817241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:30.817739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:30.819334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:30.833018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 173/340] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:30.838222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:30.868277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2ed8b-77e8-4fe8-9b02-793277912390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:30.868930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2ed8b-77e8-4fe8-9b02-793277912390 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:30.869217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2ed8b-77e8-4fe8-9b02-793277912390 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:30.869481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2ed8b-77e8-4fe8-9b02-793277912390 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:30.885832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2ed8b-77e8-4fe8-9b02-793277912390 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-99a2ed8b-77e8-4fe8-9b02-793277912390 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:30.896799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:30.897703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:30.898484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 168/341] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:30.899700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd915d8-5590-4bc7-b03b-7efd489490ba None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:30.899700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd915d8-5590-4bc7-b03b-7efd489490ba None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:30.899948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd915d8-5590-4bc7-b03b-7efd489490ba None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:30.901364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 169/342] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:30.907974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebadba-a280-4e37-bcac-da0cc184c657 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:30.907974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebadba-a280-4e37-bcac-da0cc184c657 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:30.908218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebadba-a280-4e37-bcac-da0cc184c657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:30.936172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:30.940470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ffebadba-a280-4e37-bcac-da0cc184c657 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:30.941768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ffebadba-a280-4e37-bcac-da0cc184c657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:30.982250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a683f-a7c4-4555-a3cb-97c2b7681067 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:31.009906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:31.052416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e305d-6b13-4c2b-ade7-3cabdf68f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:31.052963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e305d-6b13-4c2b-ade7-3cabdf68f319 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.053243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e305d-6b13-4c2b-ade7-3cabdf68f319 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.053514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e305d-6b13-4c2b-ade7-3cabdf68f319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.056093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 170/343] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:31.061077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebadba-a280-4e37-bcac-da0cc184c657 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:31.213503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e305d-6b13-4c2b-ade7-3cabdf68f319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:31.214998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e305d-6b13-4c2b-ade7-3cabdf68f319 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:31.271049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebadba-a280-4e37-bcac-da0cc184c657 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:31.312502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4294aaa-008a-46a0-95b0-cb12fe44d886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:31.313042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4294aaa-008a-46a0-95b0-cb12fe44d886 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.313318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4294aaa-008a-46a0-95b0-cb12fe44d886 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.313588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4294aaa-008a-46a0-95b0-cb12fe44d886 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.314254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4294aaa-008a-46a0-95b0-cb12fe44d886 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:31.314556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4294aaa-008a-46a0-95b0-cb12fe44d886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:31.315935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4294aaa-008a-46a0-95b0-cb12fe44d886 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:31.341037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 171/344] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:31.343085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1f9ff-c55c-48a3-842d-3fcb4062c9e8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.343370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1f9ff-c55c-48a3-842d-3fcb4062c9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.343646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1f9ff-c55c-48a3-842d-3fcb4062c9e8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.344971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 172/345] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:31.351906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1f9ff-c55c-48a3-842d-3fcb4062c9e8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:31.359401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:31.422569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 174/346] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:31.424021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59e7cf-f03e-4539-b591-e5a790b2c1c0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.424343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59e7cf-f03e-4539-b591-e5a790b2c1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.424714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59e7cf-f03e-4539-b591-e5a790b2c1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.438260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:31.456401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4d59e7cf-f03e-4539-b591-e5a790b2c1c0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:31.457829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4d59e7cf-f03e-4539-b591-e5a790b2c1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:31.516608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1f9ff-c55c-48a3-842d-3fcb4062c9e8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:31.557655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 175/347] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:31.558956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ef81-643d-4646-b0d6-e5907c573c29 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.559239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ef81-643d-4646-b0d6-e5907c573c29 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.559510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947ef81-643d-4646-b0d6-e5907c573c29 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.562093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed1fd6-f5c9-4d2c-aca6-96c6d36f04bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:31.562771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed1fd6-f5c9-4d2c-aca6-96c6d36f04bc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.562997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed1fd6-f5c9-4d2c-aca6-96c6d36f04bc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.563272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed1fd6-f5c9-4d2c-aca6-96c6d36f04bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.569270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 176/348] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:31.570113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ef81-643d-4646-b0d6-e5907c573c29 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:31.649229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed1fd6-f5c9-4d2c-aca6-96c6d36f04bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:31.650700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed1fd6-f5c9-4d2c-aca6-96c6d36f04bc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:31.708595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:31.722524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:31.729401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947ef81-643d-4646-b0d6-e5907c573c29 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:31.773340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:31.773702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 173/349] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:31.773993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.774269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.774539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.775765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:31.777264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:31.779163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 177/350] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:31 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:31.779962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:31.781054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb69528-26ad-4271-b5c2-3e6172fab9be None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.781439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb69528-26ad-4271-b5c2-3e6172fab9be None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.781859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb69528-26ad-4271-b5c2-3e6172fab9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.802738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5eb69528-26ad-4271-b5c2-3e6172fab9be None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:31.804383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5eb69528-26ad-4271-b5c2-3e6172fab9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:31.891910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 174/351] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:31.893882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb69528-26ad-4271-b5c2-3e6172fab9be None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:31.905116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d5b54-0888-4f3c-934e-9760214dfcff None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:31.942104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:31.942729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:31.943064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:31.943334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:31.954446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:31.960654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:31.961755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:31.962730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 178/352] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:31.963384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:31.999078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.000925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.033874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb69528-26ad-4271-b5c2-3e6172fab9be None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.072289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.072935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.075241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.075241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.081825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:32.083345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:32.086111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 175/353] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:31 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:32.086802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.095209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4b71e-0ab8-40f4-bd9c-fa5460c355c3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.150290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17532a5e-e636-4b07-b436-cbf320a15576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.150290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17532a5e-e636-4b07-b436-cbf320a15576 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.150290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17532a5e-e636-4b07-b436-cbf320a15576 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.150290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17532a5e-e636-4b07-b436-cbf320a15576 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.151517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17532a5e-e636-4b07-b436-cbf320a15576 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:32.154251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17532a5e-e636-4b07-b436-cbf320a15576 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:32.156198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 179/354] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:31 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:32.161213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154668b-1545-470a-88ec-b313ef7b17b5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.161710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154668b-1545-470a-88ec-b313ef7b17b5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.162258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154668b-1545-470a-88ec-b313ef7b17b5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.164090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 180/355] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:32.165016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154668b-1545-470a-88ec-b313ef7b17b5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.225962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee296d-bd96-4d56-97cb-7d209bfca6d4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.264871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.265504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.265777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.266040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.274516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.279705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:32.286114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:32.287022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 05 16:48:32.287798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:32.289238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.289310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 176/356] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:32.289310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.295027 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154668b-1545-470a-88ec-b313ef7b17b5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.332293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.334760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.335042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.335313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.335976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:32.337481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:32.337688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:32.346241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 181/357] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:32.347064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.407999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ea8b1-5498-423d-b391-e27ccb76e625 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.447502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c3d05-1b2f-4a15-9e10-554c240309e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.448137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3d05-1b2f-4a15-9e10-554c240309e4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.448435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3d05-1b2f-4a15-9e10-554c240309e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.448707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3d05-1b2f-4a15-9e10-554c240309e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.449439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3d05-1b2f-4a15-9e10-554c240309e4 None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=8567939112d5422e96682571a1014ec3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:32.449746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3d05-1b2f-4a15-9e10-554c240309e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:32.451195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3d05-1b2f-4a15-9e10-554c240309e4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:32.457154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 177/358] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:32.458407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f6ec5-b808-47a1-a1d3-27935c031b4b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.458694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f6ec5-b808-47a1-a1d3-27935c031b4b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.458967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f6ec5-b808-47a1-a1d3-27935c031b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.461164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6baed-3bc3-4c0d-a030-8efea160a1b8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.485019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dc2f6ec5-b808-47a1-a1d3-27935c031b4b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:32.485698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dc2f6ec5-b808-47a1-a1d3-27935c031b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:32.516736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.520895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.520895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.520895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.523281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:32.525167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:32.527983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 182/359] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:32.529061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.562359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.568502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.585914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 178/360] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:32.592243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f6ec5-b808-47a1-a1d3-27935c031b4b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.665014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd611f-5e3c-47e8-a52d-82835317aa89 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.713888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.714549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.714833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.715109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.724998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f6ec5-b808-47a1-a1d3-27935c031b4b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.732947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:32.739939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:32.742071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:32.742855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:32.744978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 183/361] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:32.746161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.774068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.775320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.775699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.776114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.776911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:32.777359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:32.779451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:32.792697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 179/362] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:32.796190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:32.868039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.892519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:32.896278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03787f55-351e-4f7f-820b-4e2bcf136e01 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.943683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba92daa-9ac1-402f-b6f0-ee319764e0bd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:32.950318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.950948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.951224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.951491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.990735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:32.991359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:32.991761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:32.992032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:32.992803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=8567939112d5422e96682571a1014ec3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:32.993206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:32.996046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.000839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 180/363] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:33.001660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.045323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.047238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.111482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec681b9-97c5-4922-ad5b-02d1c163148f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.130306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:33.148778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.149377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.149651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.149916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.150576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:33.150875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.152289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.159758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:33.161316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 181/364] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:33.162006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.186733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 184/365] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:33.187844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.288600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3480971-1e82-45e0-9022-4307843a4837 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.318813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1bf5a-5e49-4ba2-96c1-abc9b57a2165 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.327734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.328423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.328697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.328965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] PATH_INFO: `/v3/users/0fad2b0026e84e08b666772ac0cd58f3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.330143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=0fad2b0026e84e08b666772ac0cd58f3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.331682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.333989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 182/366] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/users/0fad2b0026e84e08b666772ac0cd58f3 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:33.334846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.357503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.358148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.358420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.358690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.380221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:33.382219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:33.383088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:33.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:33.384505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 185/367] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:33.385064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.399917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:33.442025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:33.458293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caee96-18a1-448d-86f2-64d28d598fb2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.495260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.495871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.496165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.496438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.497315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5d3c8-651d-4d15-9077-950f3e4d35c6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.532639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.533275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.533548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.533819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.534940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.536612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.538585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 186/368] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:33.539405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.569993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.571453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.620414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:33.652670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f17ef5-8883-48db-950f-15df29c8dde3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.672760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:33.688862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 183/369] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:33.689942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.698152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07390de-377a-4895-8300-b19c784402ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.698825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07390de-377a-4895-8300-b19c784402ff None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.699154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07390de-377a-4895-8300-b19c784402ff None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.699427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07390de-377a-4895-8300-b19c784402ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.700247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07390de-377a-4895-8300-b19c784402ff None admin] RBAC: Adding query filter params (role.id=e9db72a727fb4cd1a83a7cd4965c51a3, user.id=0fad2b0026e84e08b666772ac0cd58f3, scope.system=all) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:33.700555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07390de-377a-4895-8300-b19c784402ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.702009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07390de-377a-4895-8300-b19c784402ff None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.726750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 187/370] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/role_assignments?user.id=0fad2b0026e84e08b666772ac0cd58f3&scope.system=all&role.id=e9db72a727fb4cd1a83a7cd4965c51a3 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:33.727613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07390de-377a-4895-8300-b19c784402ff None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.805860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0558565-5308-4471-ba01-8f132b33aeb4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.848008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.848791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.849109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.849382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.850044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:33.850341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.851698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.861602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 184/371] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:33.862464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.873382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07390de-377a-4895-8300-b19c784402ff None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:33.882485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:33.924870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:33.926015 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:33.926594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:33.927060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:33.928925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:33.930469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:33.932208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 188/372] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:33.934871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:33.999019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:34.014468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5429a-6ce9-49db-92c5-416a808c0298 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.054738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.055806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.056621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.057063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.058806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.061173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.063775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 185/373] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:34.065203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:34.074082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5935b54-1537-45af-b820-3f5a13c16fda None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.117031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.117666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.117997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.118275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.166076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:34.194135 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57aa122-1880-43fd-b646-c0a3598a4a39 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.201623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.203505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.235649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.236846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.237265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.237812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.238933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=7f1cf0869430492bb74c0f643983bd5b) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:34.239398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.241736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.249205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 186/374] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:34 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:34.250253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:34.259535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:34.348882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 189/375] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:34.374837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff8527-166f-453a-990f-6ee5a1d121ab None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.380178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:34.417111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17320899-8461-4624-82b1-1c072c15701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.417736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17320899-8461-4624-82b1-1c072c15701a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.418014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17320899-8461-4624-82b1-1c072c15701a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.418285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17320899-8461-4624-82b1-1c072c15701a None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.419572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17320899-8461-4624-82b1-1c072c15701a None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.421094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17320899-8461-4624-82b1-1c072c15701a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.423006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 187/376] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:34 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:34.428820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17320899-8461-4624-82b1-1c072c15701a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:34.448431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:34.510771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d9139-5e61-4bc7-afa0-087464653e86 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.552123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.552870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.553154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.553428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] PATH_INFO: `/v3/projects/831cdea88c914362aff075e5d457eeb4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.559790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] RBAC: Authorizing `identity:get_project(project_id=831cdea88c914362aff075e5d457eeb4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.561526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.564270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 190/377] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:34 2022] GET /identity/v3/projects/831cdea88c914362aff075e5d457eeb4 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:34.571462 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:34.590415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17320899-8461-4624-82b1-1c072c15701a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.591900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:34.658811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e1ef9-9263-47ae-845d-604c3dbcb700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.659596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1ef9-9263-47ae-845d-604c3dbcb700 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.659874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1ef9-9263-47ae-845d-604c3dbcb700 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.660726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1ef9-9263-47ae-845d-604c3dbcb700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.661468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1ef9-9263-47ae-845d-604c3dbcb700 None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=7f1cf0869430492bb74c0f643983bd5b) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:34.661781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1ef9-9263-47ae-845d-604c3dbcb700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.663483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1ef9-9263-47ae-845d-604c3dbcb700 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.670720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 188/378] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:34 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:34.729505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a7fd9-7021-4b90-bb06-55e7a6fa40bf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.752980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864ee3e-f485-44ad-8f8b-fb5b372d505f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.753323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864ee3e-f485-44ad-8f8b-fb5b372d505f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.753514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864ee3e-f485-44ad-8f8b-fb5b372d505f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.755738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 189/379] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:34.761234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7791-f4d0-427b-8be6-8a048ea7f943 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.761517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7791-f4d0-427b-8be6-8a048ea7f943 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.761787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b7791-f4d0-427b-8be6-8a048ea7f943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.775323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.776370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.776839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.777312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] PATH_INFO: `/v3/groups/b0735de1876f4a00ab1ac44dda6f66a8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.783299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d74b7791-f4d0-427b-8be6-8a048ea7f943 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:34.784528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d74b7791-f4d0-427b-8be6-8a048ea7f943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:34.786747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] RBAC: Authorizing `identity:get_group(group_id=b0735de1876f4a00ab1ac44dda6f66a8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.788469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.790657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 191/380] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:34 2022] GET /identity/v3/groups/b0735de1876f4a00ab1ac44dda6f66a8 => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:34.798228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:34.866185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 190/381] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:34.903381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:34.944266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6781c-a1ae-462a-b7ea-c0dd009ed96d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:34.988793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c81c95-5dda-49f0-8cbc-fb14390fd6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:34.989762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c81c95-5dda-49f0-8cbc-fb14390fd6ef None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:34.990230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c81c95-5dda-49f0-8cbc-fb14390fd6ef None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:34.990719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c81c95-5dda-49f0-8cbc-fb14390fd6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:34.991630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c81c95-5dda-49f0-8cbc-fb14390fd6ef None admin] RBAC: Adding query filter params (group.id=b0735de1876f4a00ab1ac44dda6f66a8, role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=831cdea88c914362aff075e5d457eeb4) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:34.992122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c81c95-5dda-49f0-8cbc-fb14390fd6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:34.993774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c81c95-5dda-49f0-8cbc-fb14390fd6ef None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:34.999712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 192/382] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:34 2022] GET /identity/v3/role_assignments?group.id=b0735de1876f4a00ab1ac44dda6f66a8&scope.project.id=831cdea88c914362aff075e5d457eeb4&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:35.183168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b7791-f4d0-427b-8be6-8a048ea7f943 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:35.187499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:35.300404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b7791-f4d0-427b-8be6-8a048ea7f943 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:35.344815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:35.345798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:35.346297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:35.346754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:35.364589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205869-da97-4f8c-ad01-6d706c3beedf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:35.364908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205869-da97-4f8c-ad01-6d706c3beedf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:35.365117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32205869-da97-4f8c-ad01-6d706c3beedf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:35.366866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 193/383] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:35.372515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272e35c-b600-41fe-ad32-0f87ddf91ed3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:35.372661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272e35c-b600-41fe-ad32-0f87ddf91ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:35.372866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272e35c-b600-41fe-ad32-0f87ddf91ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:35.393155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:35.398543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:35.399391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:35.400235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:35.406906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6272e35c-b600-41fe-ad32-0f87ddf91ed3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:35.407162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 191/384] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:35.408653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6272e35c-b600-41fe-ad32-0f87ddf91ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:35.411480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:35.514368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 194/385] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:35.557237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:35.603710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b0d9f-0baa-4dd4-82f7-2cce39b9767a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:35.645183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:35.645907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:35.646190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:35.646464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:35.647130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:35.647436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:35.648835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:35.660767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 192/386] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:35.666948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272e35c-b600-41fe-ad32-0f87ddf91ed3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:35.764553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:35.804079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272e35c-b600-41fe-ad32-0f87ddf91ed3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:35.841393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:35.842059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:35.848700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:35.849037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:35.850345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:35.851907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:35.853893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 195/387] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:35 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:35.860974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:35.996457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad6a14-a9f6-440b-bb1b-41e8c5c95b7d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:36.025128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:36.033020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:36.044009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:36.044676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.044956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.045230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:36.048832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:36.051972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:36.052319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 193/388] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:35 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:36.058735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:36.190662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaf8d5-107c-49ca-9643-a2e7f6811203 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:36.228499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:36.229475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.229899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.230313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:36.233151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] RBAC: Authorizing `identity:create_grant(project_id=8567939112d5422e96682571a1014ec3, group_id=44e32e38f7e241f6932b8713c2b8d1bf, role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:36.234812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:36.262555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31b62-b71c-400b-91af-f84fcdbfbc76 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:36.271238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 194/389] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Sep 5 16:48:36 2022] PUT /identity/v3/projects/8567939112d5422e96682571a1014ec3/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/18b82b06d87e4c93bef42bca94fc658d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:48:36.317109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:36.317769 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.318046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.318314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:36.327032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:36.338181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:36.343128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:36.344050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:36.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 196/390] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:36.350521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:36.479502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:36.535211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59abfd2-0362-449b-887e-f4cf46f5b1bf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:36.571836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:36.572477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.572763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.573018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:36.573675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:36.573974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:36.575395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:36.586250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 195/391] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:36.591444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:36.733312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd9000-aae3-40d2-8da0-87e2229da684 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:36.779295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:36.780304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.780764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.781209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:36.782515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:36.784130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:36.786675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 197/392] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:36 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:36.791833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:36.911355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844b949-8c26-4a41-9209-bf6a987c83d7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:36.932496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:36.953894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4351aa-ab48-4062-b307-96d6d1a26a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:36.954528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4351aa-ab48-4062-b307-96d6d1a26a25 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.954801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4351aa-ab48-4062-b307-96d6d1a26a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.955068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4351aa-ab48-4062-b307-96d6d1a26a25 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:36.956182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4351aa-ab48-4062-b307-96d6d1a26a25 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:36.957751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4351aa-ab48-4062-b307-96d6d1a26a25 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:36.959851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 196/393] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:36 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:36.967289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:36.969360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65a224-6445-4bdc-b2f8-5c6bae06b552 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.969639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65a224-6445-4bdc-b2f8-5c6bae06b552 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.969907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65a224-6445-4bdc-b2f8-5c6bae06b552 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:36.971320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 197/394] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:36.976798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63eec-fc20-4bf1-a837-f66733d0040e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:36.977079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63eec-fc20-4bf1-a837-f66733d0040e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:36.977356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da63eec-fc20-4bf1-a837-f66733d0040e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.010306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3da63eec-fc20-4bf1-a837-f66733d0040e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:37.011423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3da63eec-fc20-4bf1-a837-f66733d0040e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:37.082477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 198/395] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:37.117260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:37.178399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882a4d-57bd-4ae6-965a-8d301d63852e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:37.179166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e922d8a-73d2-4e30-b0a4-318e924c58c9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.179590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e922d8a-73d2-4e30-b0a4-318e924c58c9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.179994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e922d8a-73d2-4e30-b0a4-318e924c58c9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.181522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 199/396] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:37.186778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc6062-4d83-4926-bc70-69a0d1b1bf62 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.187062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc6062-4d83-4926-bc70-69a0d1b1bf62 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.187335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc6062-4d83-4926-bc70-69a0d1b1bf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.219245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9dcc6062-4d83-4926-bc70-69a0d1b1bf62 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:37.222029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9dcc6062-4d83-4926-bc70-69a0d1b1bf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:37.223396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:37.225487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.226011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.226464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.230789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] RBAC: Authorizing `identity:create_grant(project_id=8567939112d5422e96682571a1014ec3, group_id=44e32e38f7e241f6932b8713c2b8d1bf, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:37.232662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:37.255949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 198/397] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Sep 5 16:48:36 2022] PUT /identity/v3/projects/8567939112d5422e96682571a1014ec3/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:37.320596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 200/398] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:37.408417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:37.497905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617ec6d-2fa7-4640-9ce3-056c036738fc None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.498474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617ec6d-2fa7-4640-9ce3-056c036738fc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.499013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617ec6d-2fa7-4640-9ce3-056c036738fc None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.500572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 201/399] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:37.507405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945b307-ec38-4942-907a-d3805e2b8829 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.507971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945b307-ec38-4942-907a-d3805e2b8829 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.508489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945b307-ec38-4942-907a-d3805e2b8829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.522569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041fb9d-6031-4aec-8504-cd4d3b6967eb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:37.529355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4945b307-ec38-4942-907a-d3805e2b8829 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:37.530836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4945b307-ec38-4942-907a-d3805e2b8829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:37.550798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:37.566468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:37.567114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.567390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.567657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.580304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:37.587829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:37.589287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:37.590429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:37.591481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 199/400] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:37.597540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:37.626790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 202/401] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:37.712268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045ed45-8edd-4af4-ba00-a302cc527692 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:37.735489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945b307-ec38-4942-907a-d3805e2b8829 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:37.760834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:37.763146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.763625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.764182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.766257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:37.766778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:37.770642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:37.787645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 200/402] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:37.793630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:37.888634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:37.925485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945b307-ec38-4942-907a-d3805e2b8829 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:37.976974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:37.977932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:37.978353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:37.978748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:37.989680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bf67d-ea6d-41f7-81aa-c52d59ecbac9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:37.992951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:37.998163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:37.999037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:37.999824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:38.001210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 203/403] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:38.001210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.025761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.026483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.026823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.027163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.028470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.029989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.032124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 201/404] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:37 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:38.032961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.045230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:38.111689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73ccf7-1fc1-4ace-bd5d-0e705e8a2613 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.155481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.156389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.156821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.157245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.161248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5aaf2-d9f2-4598-b459-3cba3834d56d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.169300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:38.174640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:38.175670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:38.177148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:38.178130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:38.180457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 204/405] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:38.181521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.203268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.203922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.204164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.204455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.205111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:38.205410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.206792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.215138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 202/406] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:38.215938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.288718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:38.296675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a981e14-e573-43af-8736-68d4bfbd3715 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.317822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d59013-51a8-45ad-8981-6796763578d3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.335202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8791-e903-4451-8452-0915946515af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.336070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8791-e903-4451-8452-0915946515af None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.336519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8791-e903-4451-8452-0915946515af None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.336931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c8791-e903-4451-8452-0915946515af None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.338309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8791-e903-4451-8452-0915946515af None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.339895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c8791-e903-4451-8452-0915946515af None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.346618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 205/407] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:38 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:38.347734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8791-e903-4451-8452-0915946515af None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.352187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.352968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.352968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.353212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.353873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:38.354173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.355970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.364989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 203/408] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:38.365793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.407051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:38.473383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c8791-e903-4451-8452-0915946515af None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.475895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c480868-b519-4ee9-a0cc-9f2b95cbb41c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.510121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29413059-213c-4095-a763-817067ae7b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.510790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29413059-213c-4095-a763-817067ae7b46 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.511067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29413059-213c-4095-a763-817067ae7b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.511332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29413059-213c-4095-a763-817067ae7b46 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.512591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.513397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.513518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29413059-213c-4095-a763-817067ae7b46 None admin] RBAC: Authorizing `identity:create_grant(project_id=7f1cf0869430492bb74c0f643983bd5b, group_id=44e32e38f7e241f6932b8713c2b8d1bf, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.513916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.514319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.514930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29413059-213c-4095-a763-817067ae7b46 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.515699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.517412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.519386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 206/409] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:38 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:38.520554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.538777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 204/410] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Sep 5 16:48:38 2022] PUT /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:38.540434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29413059-213c-4095-a763-817067ae7b46 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.566950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:38.718283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc3e13-cb32-40ad-8bea-7c2297b3cd9a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.732505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:38.738619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29413059-213c-4095-a763-817067ae7b46 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:38.774965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b5107-8722-46ad-8f04-67ec98affca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.775632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b5107-8722-46ad-8f04-67ec98affca7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.775918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b5107-8722-46ad-8f04-67ec98affca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.776482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b5107-8722-46ad-8f04-67ec98affca7 None admin] PATH_INFO: `/v3/groups/b0735de1876f4a00ab1ac44dda6f66a8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.778238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b5107-8722-46ad-8f04-67ec98affca7 None admin] RBAC: Authorizing `identity:get_group(group_id=b0735de1876f4a00ab1ac44dda6f66a8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.780100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b5107-8722-46ad-8f04-67ec98affca7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.782519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 207/411] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:38 2022] GET /identity/v3/groups/b0735de1876f4a00ab1ac44dda6f66a8 => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:38.788495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc922d-9796-4611-bcc5-e81d75119cbb None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.788802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc922d-9796-4611-bcc5-e81d75119cbb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.789083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc922d-9796-4611-bcc5-e81d75119cbb None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.790206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26a773-cfdc-4f29-bf7d-8a433dc4247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:38.790652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 208/412] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:38.790845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a773-cfdc-4f29-bf7d-8a433dc4247e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.791123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a773-cfdc-4f29-bf7d-8a433dc4247e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.791390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc922d-9796-4611-bcc5-e81d75119cbb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.791500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26a773-cfdc-4f29-bf7d-8a433dc4247e None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.793022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26a773-cfdc-4f29-bf7d-8a433dc4247e None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:38.794460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26a773-cfdc-4f29-bf7d-8a433dc4247e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:38.796255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 205/413] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:38 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:38.797361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8ea2-64f1-43ba-8a73-6dfb19030c44 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:38.797687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8ea2-64f1-43ba-8a73-6dfb19030c44 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:38.797958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8ea2-64f1-43ba-8a73-6dfb19030c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:38.817882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0c2f8ea2-64f1-43ba-8a73-6dfb19030c44 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:38.819183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0c2f8ea2-64f1-43ba-8a73-6dfb19030c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:38.902845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 206/414] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:38.904000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f8ea2-64f1-43ba-8a73-6dfb19030c44 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:38.933968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:38.945998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc922d-9796-4611-bcc5-e81d75119cbb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:39.029885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:39.030450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.030729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.031003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] PATH_INFO: `/v3/projects/831cdea88c914362aff075e5d457eeb4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.032726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:39.036900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] RBAC: Authorizing `identity:get_project(project_id=831cdea88c914362aff075e5d457eeb4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:39.038405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:39.040191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 209/415] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:38 2022] GET /identity/v3/projects/831cdea88c914362aff075e5d457eeb4 => generated 344 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:39.041610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f8ea2-64f1-43ba-8a73-6dfb19030c44 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:39.045610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:39.078550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:39.079538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.079997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.080448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.083137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] RBAC: Authorizing `identity:create_grant(project_id=7f1cf0869430492bb74c0f643983bd5b, group_id=44e32e38f7e241f6932b8713c2b8d1bf, role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:39.084832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:39.100617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 207/416] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Sep 5 16:48:38 2022] PUT /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b/groups/44e32e38f7e241f6932b8713c2b8d1bf/roles/18b82b06d87e4c93bef42bca94fc658d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:48:39.225093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:39.237734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb704ba-2413-4302-8731-2a070e242d9f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:39.285595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:39.286190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.286468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.286803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] PATH_INFO: `/v3/projects/831cdea88c914362aff075e5d457eeb4/groups/b0735de1876f4a00ab1ac44dda6f66a8/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.290339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] RBAC: Authorizing `identity:create_grant(project_id=831cdea88c914362aff075e5d457eeb4, group_id=b0735de1876f4a00ab1ac44dda6f66a8, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:39.291782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:39.308876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 210/417] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Sep 5 16:48:39 2022] PUT /identity/v3/projects/831cdea88c914362aff075e5d457eeb4/groups/b0735de1876f4a00ab1ac44dda6f66a8/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:48:39.349990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90657ba3-5c90-413d-a8ee-8a9fc8a85f2b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:39.392825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:39.393848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.394385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.394991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.408116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:39.413990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:39.414934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:39.415740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:39.417712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 208/418] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:39.423844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:39.473642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:39.553069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f235365-4740-49e2-a756-7f3207aa8268 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:39.588660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83499d7e-7a66-4356-bc99-1e598bf9c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:39.589622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83499d7e-7a66-4356-bc99-1e598bf9c39a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.590106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83499d7e-7a66-4356-bc99-1e598bf9c39a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.590571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83499d7e-7a66-4356-bc99-1e598bf9c39a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.591532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83499d7e-7a66-4356-bc99-1e598bf9c39a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:39.592031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83499d7e-7a66-4356-bc99-1e598bf9c39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:39.595101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83499d7e-7a66-4356-bc99-1e598bf9c39a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:39.604407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 211/419] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:39.610645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:39.643829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:39.684598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93c57-e90d-4275-b599-6af490e00552 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.685613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93c57-e90d-4275-b599-6af490e00552 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.686112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93c57-e90d-4275-b599-6af490e00552 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.687959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 212/420] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:39.700653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324caf1-10d1-4bcd-8398-321686e4c57b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.701268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324caf1-10d1-4bcd-8398-321686e4c57b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.701743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324caf1-10d1-4bcd-8398-321686e4c57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.734509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8324caf1-10d1-4bcd-8398-321686e4c57b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:39.735707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8324caf1-10d1-4bcd-8398-321686e4c57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:39.781627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12386f23-3315-40ac-ae3b-0017867223ba None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:39.843544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:39.844179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:39.844470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:39.844747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:39.875870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 213/421] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:39.923970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:39.925469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:39.940441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:40.025398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 209/422] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:40.031025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324caf1-10d1-4bcd-8398-321686e4c57b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:40.065518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:40.179149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324caf1-10d1-4bcd-8398-321686e4c57b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:40.227229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:40.227927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.228289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.228564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:40.229743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:40.231506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:40.233481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 214/423] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:40 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:40.239211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:40.276597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:40.359448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a8fa9-7622-4f3b-97c2-52b4ad8029d7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:40.376234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:40.399011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:40.399827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.400237 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.400854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:40.402192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:40.403763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:40.406712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 210/424] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:40 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:40.415081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:40.461327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:40.502091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1249b4-8304-4530-9f0f-f325c3fcd3d5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:40.537845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8d2b3-b258-4202-b974-05446d82baae None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:40.567800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:40.568470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.568742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.569005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:40.584716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:40.593494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:40.594358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:40.595137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 215/425] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:40.602921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:40.609233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:40.609853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.610125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.610390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:40.611399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=8567939112d5422e96682571a1014ec3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:40.611705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:40.616773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:40.627319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 211/426] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:40 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:40.732353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e645e17-2fde-4bf7-8908-bf6c9fba4cd4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:40.775589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2098-72d4-4ac0-b154-c10bde756c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:40.776294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2098-72d4-4ac0-b154-c10bde756c0a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.776573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2098-72d4-4ac0-b154-c10bde756c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.776841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2098-72d4-4ac0-b154-c10bde756c0a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:40.777515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc2098-72d4-4ac0-b154-c10bde756c0a None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:40.777811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc2098-72d4-4ac0-b154-c10bde756c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:40.779192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc2098-72d4-4ac0-b154-c10bde756c0a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:40.793994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 216/427] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:40.797754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:40.858066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:40.922372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627a9ee-6f19-4154-9996-575b011e1333 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:40.960816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a025688-6e06-4546-a352-966123c4fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:40.961455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025688-6e06-4546-a352-966123c4fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.961736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025688-6e06-4546-a352-966123c4fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.962012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a025688-6e06-4546-a352-966123c4fcf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:40.966808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781901-2d8b-4ce7-9be6-b446328f60ba None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.967093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781901-2d8b-4ce7-9be6-b446328f60ba None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.967362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32781901-2d8b-4ce7-9be6-b446328f60ba None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:40.968970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 217/428] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:40.976480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709273-4801-4289-a6eb-f14723a3cd49 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:40.976761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709273-4801-4289-a6eb-f14723a3cd49 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:40.977035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52709273-4801-4289-a6eb-f14723a3cd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.001015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-52709273-4801-4289-a6eb-f14723a3cd49 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:41.002111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-52709273-4801-4289-a6eb-f14723a3cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:41.004347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:41.043996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a025688-6e06-4546-a352-966123c4fcf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:41.045941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a025688-6e06-4546-a352-966123c4fcf5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:41.093964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 218/429] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:41.103552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:41.159440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 212/430] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:41.165172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52709273-4801-4289-a6eb-f14723a3cd49 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:41.268039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d779c-cd24-4f00-a6d2-6a243cb74b40 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.268596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d779c-cd24-4f00-a6d2-6a243cb74b40 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.268596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d779c-cd24-4f00-a6d2-6a243cb74b40 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.270067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 213/431] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:41.274985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea928-cbfa-411d-9f85-11e3f4e03459 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.275267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea928-cbfa-411d-9f85-11e3f4e03459 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.275576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ea928-cbfa-411d-9f85-11e3f4e03459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.278291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52709273-4801-4289-a6eb-f14723a3cd49 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:41.295542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a50ea928-cbfa-411d-9f85-11e3f4e03459 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:41.296798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a50ea928-cbfa-411d-9f85-11e3f4e03459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:41.319078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:41.319853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.320248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.320883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] PATH_INFO: `/v3/projects/8567939112d5422e96682571a1014ec3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.322327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] RBAC: Authorizing `identity:get_project(project_id=8567939112d5422e96682571a1014ec3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:41.324525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:41.330783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 219/432] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:41 2022] GET /identity/v3/projects/8567939112d5422e96682571a1014ec3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:41.333179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:41.343961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:41.384667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 214/433] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:41.408856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ea928-cbfa-411d-9f85-11e3f4e03459 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:41.456526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84682ac-8394-4c48-b9f7-08aaf214e43c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:41.499344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:41.500014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.500871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.501146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.502453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:41.505422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:41.512410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 220/434] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:41 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:41.514484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:41.515159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:41.540937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ea928-cbfa-411d-9f85-11e3f4e03459 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:41.581564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:41.582922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.583517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.584102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.597841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:41.603124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:41.604064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:41.604871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 215/435] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:41.611316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:41.619547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:41.652677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9fc4d-a6f3-4e67-9f3f-bcf1a8d1e44a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:41.690875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:41.691853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.692311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.692742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.693604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=8567939112d5422e96682571a1014ec3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:41.694054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:41.695640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:41.702074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 221/436] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:41 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=8567939112d5422e96682571a1014ec3&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:41.703019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:41.748131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bc50-6115-4ead-add2-5a800f26d792 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:41.788320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315fd9c-1ef5-4b46-b6b9-d3eff52f0d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:41.788954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fd9c-1ef5-4b46-b6b9-d3eff52f0d7a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.789295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fd9c-1ef5-4b46-b6b9-d3eff52f0d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.789569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315fd9c-1ef5-4b46-b6b9-d3eff52f0d7a None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.790270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315fd9c-1ef5-4b46-b6b9-d3eff52f0d7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:41.790579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315fd9c-1ef5-4b46-b6b9-d3eff52f0d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:41.792033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315fd9c-1ef5-4b46-b6b9-d3eff52f0d7a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:41.794694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:41.802192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 216/437] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:48:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:41.803558 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c18dd-495a-48db-8033-5835357c0361 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.803845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c18dd-495a-48db-8033-5835357c0361 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.804118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c18dd-495a-48db-8033-5835357c0361 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.809371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 217/438] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:41.809371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c18dd-495a-48db-8033-5835357c0361 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:41.838569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e01cd-6c7a-45d9-805b-d39a1c975641 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:41.881238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369ea2c-f369-4e07-a64b-f0b9e5764b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:41.882180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369ea2c-f369-4e07-a64b-f0b9e5764b68 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.882180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369ea2c-f369-4e07-a64b-f0b9e5764b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.882404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369ea2c-f369-4e07-a64b-f0b9e5764b68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.890493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:41.896659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369ea2c-f369-4e07-a64b-f0b9e5764b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a369ea2c-f369-4e07-a64b-f0b9e5764b68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:41.901650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:41.902539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 222/439] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:41.904402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865236-2d3c-45c0-ad6b-3c5a173e8b48 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.905635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865236-2d3c-45c0-ad6b-3c5a173e8b48 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.905909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c865236-2d3c-45c0-ad6b-3c5a173e8b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:41.924966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2c865236-2d3c-45c0-ad6b-3c5a173e8b48 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:41.926191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c865236-2d3c-45c0-ad6b-3c5a173e8b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:41.929522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c18dd-495a-48db-8033-5835357c0361 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:41.967235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:41.967890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:41.969582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:41.969582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.010087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 223/440] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:42.010948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c865236-2d3c-45c0-ad6b-3c5a173e8b48 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.053508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.060717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.061347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.153301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c865236-2d3c-45c0-ad6b-3c5a173e8b48 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.188195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.191371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 218/441] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:42.197135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.197299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.197991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.198330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.198609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.199252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:42.199556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.200980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.209605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 224/442] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:42.215493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.312828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.316858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b906864-b154-432e-bb48-bee23e051003 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.341256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a759d5-a5cf-4489-b4af-4ad71475bff8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.352122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.352859 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.353135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.353403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.354515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.356361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.357999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 219/443] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:42.360982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.375591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.376707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.376707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.376843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.432530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.444866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.446418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.463811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed88b2a-97dd-4149-9427-1c14535ed87c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.498747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.499372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.499651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.499927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.510576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:42.515546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:42.516659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:42.517422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 220/444] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:42.518240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.545918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.548257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 225/445] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 16:48:42.549120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.619440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09ea707-6088-4120-ba53-45afaa861782 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.635996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.645973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036982fd-8e14-44da-a9ce-2be37fa3c8b9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.652488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.653112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.653375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.653641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.654732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.656189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.657761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 221/446] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:42.658506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.684564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.685169 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.685441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.685708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.686352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:42.686647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.688003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.696645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 226/447] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:42.697412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.763233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.770912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1b6f8-a07d-487e-9cb8-bd6216889adb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.796123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44478a78-201a-4c76-ac31-ea9dc08b0018 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.805065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.805695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.805974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.806248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] PATH_INFO: `/v3/projects/7f1cf0869430492bb74c0f643983bd5b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.807362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] RBAC: Authorizing `identity:get_project(project_id=7f1cf0869430492bb74c0f643983bd5b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.808859 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.810488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 222/448] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/projects/7f1cf0869430492bb74c0f643983bd5b => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:42.811274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.834618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.835216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.835487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.835754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:42.836490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=e68472fdebed4f58b495dc466408a9e4, scope.project.id=7f1cf0869430492bb74c0f643983bd5b) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:42.836793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:42.838207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:42.843888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 227/449] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=e68472fdebed4f58b495dc466408a9e4 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:42.844849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:42.874864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:42.926619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d805f69-34b0-41ce-acaf-eaccdbd29b44 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.967530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:42.968171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:42.968277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5e8a9-4644-4aad-beed-e3042035c187 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:42.968477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:42.968748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:43.003100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:43.013595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:43.014242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:43.014664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:43.014994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] PATH_INFO: `/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:43.016286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] RBAC: Authorizing `identity:get_group(group_id=44e32e38f7e241f6932b8713c2b8d1bf)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:43.017773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:43.019774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 228/450] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/groups/44e32e38f7e241f6932b8713c2b8d1bf => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:43.026952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:43.058337 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:43.059825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:43.130726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:43.149814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c16f4-bdc3-4a2d-8240-bb14bbe81336 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:43.182501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 223/451] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:43.189944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:43.190570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:43.191288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:43.191635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:43.191972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:43.192802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] RBAC: Adding query filter params (group.id=44e32e38f7e241f6932b8713c2b8d1bf, role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=7f1cf0869430492bb74c0f643983bd5b) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:43.193854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:43.194618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:43.201237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 229/452] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:43 2022] GET /identity/v3/role_assignments?group.id=44e32e38f7e241f6932b8713c2b8d1bf&scope.project.id=7f1cf0869430492bb74c0f643983bd5b&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:43.261152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:43.303897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ec1e5-bd49-40bf-90af-b6996dd66ffc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:43.342907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:43.343566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:43.343784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:43.344052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] PATH_INFO: `/v3/projects/831cdea88c914362aff075e5d457eeb4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:43.345272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] RBAC: Authorizing `identity:get_project(project_id=831cdea88c914362aff075e5d457eeb4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:43.346727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:43.348559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 224/453] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:48:43 2022] GET /identity/v3/projects/831cdea88c914362aff075e5d457eeb4 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:43.355790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:43.463985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a896db-1d07-42b0-86da-6f3e04bba5dc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:43.499561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b253462-0410-44da-bf52-79ce825bb8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:43.500683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b253462-0410-44da-bf52-79ce825bb8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:43.501168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b253462-0410-44da-bf52-79ce825bb8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:43.501652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b253462-0410-44da-bf52-79ce825bb8f0 None admin] PATH_INFO: `/v3/groups/b0735de1876f4a00ab1ac44dda6f66a8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:43.503542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b253462-0410-44da-bf52-79ce825bb8f0 None admin] RBAC: Authorizing `identity:get_group(group_id=b0735de1876f4a00ab1ac44dda6f66a8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:43.505913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b253462-0410-44da-bf52-79ce825bb8f0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:43.507747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 230/454] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Sep 5 16:48:43 2022] GET /identity/v3/groups/b0735de1876f4a00ab1ac44dda6f66a8 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:43.513546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:43.628442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da2462-25aa-4593-b2b4-7163c3d7a474 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:43.658658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:43.668500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dbed7-e186-4bdd-936b-6b7209c7c919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:43.669394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dbed7-e186-4bdd-936b-6b7209c7c919 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:43.669707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dbed7-e186-4bdd-936b-6b7209c7c919 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:43.670041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dbed7-e186-4bdd-936b-6b7209c7c919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:43.671149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dbed7-e186-4bdd-936b-6b7209c7c919 None admin] RBAC: Adding query filter params (group.id=b0735de1876f4a00ab1ac44dda6f66a8, role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=831cdea88c914362aff075e5d457eeb4) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:43.671548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dbed7-e186-4bdd-936b-6b7209c7c919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:43.673828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dbed7-e186-4bdd-936b-6b7209c7c919 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:43.683590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 225/455] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Sep 5 16:48:43 2022] GET /identity/v3/role_assignments?group.id=b0735de1876f4a00ab1ac44dda6f66a8&scope.project.id=831cdea88c914362aff075e5d457eeb4&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:45.916914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1808900-70e9-4f7b-8ff4-8adee744e708 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:45.917459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1808900-70e9-4f7b-8ff4-8adee744e708 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:45.917459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1808900-70e9-4f7b-8ff4-8adee744e708 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:45.919045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 231/456] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:45.925167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa85e12-75f6-4b50-a825-bbba64689749 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:45.925955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa85e12-75f6-4b50-a825-bbba64689749 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:45.925955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa85e12-75f6-4b50-a825-bbba64689749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:45.950761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-caa85e12-75f6-4b50-a825-bbba64689749 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:45.951966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-caa85e12-75f6-4b50-a825-bbba64689749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:46.037690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 226/457] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:46.124727 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801eb1-b30b-4a99-9141-0afbd1b62c4c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.125373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801eb1-b30b-4a99-9141-0afbd1b62c4c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.125840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801eb1-b30b-4a99-9141-0afbd1b62c4c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.127643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 232/458] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:46.132876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2b96d-b942-4a2c-8609-7abcbf738def None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.133410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2b96d-b942-4a2c-8609-7abcbf738def None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.133899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2b96d-b942-4a2c-8609-7abcbf738def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.161166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b3f2b96d-b942-4a2c-8609-7abcbf738def None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:46.162682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b3f2b96d-b942-4a2c-8609-7abcbf738def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:46.231470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 227/459] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:46.315141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ebd5d-ee3e-48ff-81f0-e508f64c7547 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.315774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ebd5d-ee3e-48ff-81f0-e508f64c7547 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.316385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ebd5d-ee3e-48ff-81f0-e508f64c7547 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.318273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 233/460] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:46.323301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32514473-b927-47d2-b07c-f2b2a90abc62 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.323850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32514473-b927-47d2-b07c-f2b2a90abc62 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.324343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32514473-b927-47d2-b07c-f2b2a90abc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.349119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-32514473-b927-47d2-b07c-f2b2a90abc62 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:46.350689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-32514473-b927-47d2-b07c-f2b2a90abc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:46.413695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:46.436368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 228/461] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:46.466583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ebd5d-ee3e-48ff-81f0-e508f64c7547 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:46.493399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:46.542199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32514473-b927-47d2-b07c-f2b2a90abc62 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:46.599270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ebd5d-ee3e-48ff-81f0-e508f64c7547 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:46.653437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:46.654082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.654358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.654623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.673365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:46.674818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:46.702828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:46.703791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:46.704684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:46.704684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:46.704684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:46.712225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 234/462] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:46.713108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:46.759760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32514473-b927-47d2-b07c-f2b2a90abc62 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:46.796694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ab1e7-4c14-45aa-b03f-c428df0f4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:46.797336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ab1e7-4c14-45aa-b03f-c428df0f4b83 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.797628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ab1e7-4c14-45aa-b03f-c428df0f4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.797886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ab1e7-4c14-45aa-b03f-c428df0f4b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.807781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:46.809049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ab1e7-4c14-45aa-b03f-c428df0f4b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:46.810676 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ab1e7-4c14-45aa-b03f-c428df0f4b83 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-658ab1e7-4c14-45aa-b03f-c428df0f4b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:46.826151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:46.827088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:46.827911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:46.827911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:46.827911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:46.827911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 229/463] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:46.828913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3443-4183-4206-a2f9-811f73c8a803 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.828913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3443-4183-4206-a2f9-811f73c8a803 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.829054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b3443-4183-4206-a2f9-811f73c8a803 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.830352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 230/464] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:46.833115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3443-4183-4206-a2f9-811f73c8a803 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:46.851172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025d96c-905f-432e-bf50-8c39db66df52 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:46.873202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:46.887387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:46.887991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.888296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.888568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.889204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:46.889502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:46.890898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:46.904831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 235/465] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:46.905548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:46.944523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b3443-4183-4206-a2f9-811f73c8a803 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:46.978192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96964e2d-b98f-4ee2-b0f1-1282be11d7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:46.978823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964e2d-b98f-4ee2-b0f1-1282be11d7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:46.979096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964e2d-b98f-4ee2-b0f1-1282be11d7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:46.979526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964e2d-b98f-4ee2-b0f1-1282be11d7d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:46.990619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96964e2d-b98f-4ee2-b0f1-1282be11d7d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:46.992071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96964e2d-b98f-4ee2-b0f1-1282be11d7d8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-96964e2d-b98f-4ee2-b0f1-1282be11d7d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:47.006481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:47.007374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:47.008227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:47.008227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:47.008227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:47.008227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 231/466] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:47.008937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f8c94-4728-47c6-a903-e454ccefc1ad None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.009207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f8c94-4728-47c6-a903-e454ccefc1ad None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.009473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f8c94-4728-47c6-a903-e454ccefc1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.021021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3c4fb-d123-425b-8dac-478006c9a7b2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:47.027964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d57f8c94-4728-47c6-a903-e454ccefc1ad None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:47.029166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d57f8c94-4728-47c6-a903-e454ccefc1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:47.047910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:47.057466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:47.058075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.058356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.058629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.059268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:47.059575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:47.060991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.072505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 236/467] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:47.077667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:47.112172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 232/468] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:47.112172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f8c94-4728-47c6-a903-e454ccefc1ad None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:47.130623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:47.193374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd2006-b52b-42c5-80f6-a3663323d292 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:47.228711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f8c94-4728-47c6-a903-e454ccefc1ad None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:47.233718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:47.234424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.234766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.235139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.235845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:47.237851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.268082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 237/469] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:47 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:47.268989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:47.269165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:47.269685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.270017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.270336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.271100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:47.271401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:47.273291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.287595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 233/470] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:47.293397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:47.326579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:47.397002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea23da-2e40-4e0d-8ea0-767fc3b470f9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:47.437208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:47.437835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.438109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.438376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.439025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:47.440481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.470374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 238/471] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:47 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:47.471136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:47.479403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:47.507910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5bc41-b1e4-4056-8df6-da4b70fc4373 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:47.554403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:47.554952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.555230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.555501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.556175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:47.557687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.587958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 234/472] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:47 2022] POST /identity/v3/users => generated 248 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:47.621098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:47.626212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71394b67-ff3e-40cf-9f7a-13821c6b7971 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:47.669269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6a89f-daa3-4e50-aad9-a23bb0510b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:47.669933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6a89f-daa3-4e50-aad9-a23bb0510b40 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.670206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6a89f-daa3-4e50-aad9-a23bb0510b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.670475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6a89f-daa3-4e50-aad9-a23bb0510b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.683324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6a89f-daa3-4e50-aad9-a23bb0510b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:47.684838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6a89f-daa3-4e50-aad9-a23bb0510b40 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a0d6a89f-daa3-4e50-aad9-a23bb0510b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:47.701703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:47.702650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:47.703462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:47.703462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:47.703462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:47.703462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 239/473] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:47.711265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:47.828366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38479ec-f59a-475d-a359-8d311f67c330 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.828746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38479ec-f59a-475d-a359-8d311f67c330 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.828969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38479ec-f59a-475d-a359-8d311f67c330 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.830348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 240/474] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:47.834803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc55dc-1604-4599-b750-767c0449c198 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:47.836671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9797e1-7e24-45bb-a259-7806c804a1ed None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.836960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9797e1-7e24-45bb-a259-7806c804a1ed None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.837235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9797e1-7e24-45bb-a259-7806c804a1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.858504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5b9797e1-7e24-45bb-a259-7806c804a1ed None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:47.860772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5b9797e1-7e24-45bb-a259-7806c804a1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:47.878971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:47.879683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:47.880017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:47.880328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:47.880980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:47.881284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:47.883660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:47.885453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:47.903660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 235/475] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:47.910288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:47.951717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 241/476] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:48.022764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7933ece-1344-4751-8ccf-9bde732e8426 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:48.058638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:48.059514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:48.059933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:48.060370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:48.061148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:48.062740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:48.090880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 236/477] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:47 2022] POST /identity/v3/users => generated 249 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:48.232114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9797e1-7e24-45bb-a259-7806c804a1ed None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:48.320378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:48.349442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9797e1-7e24-45bb-a259-7806c804a1ed None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:48.396201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:48.396831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:48.397111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:48.397385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:48.398123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:48.399563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:48.411089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.sql.core [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 16:48:48.411089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 16:48:48.411089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [parameters: {'id': 'afb9dde57625409e895f28ac5b426c6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 16:48:48.411089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81522) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application conn.query(q) Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result.read() Sep 05 16:48:48.429177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return ref.to_dict() Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 16:48:48.430080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application next(self.gen) Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application yield resource Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application next(self.gen) Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise self.value Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application session.commit() Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 16:48:48.430971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.session.flush() Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application compat.raise_( Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 16:48:48.431818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 16:48:48.432734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application conn.query(q) Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result.read() Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 16:48:48.434932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [parameters: {'id': 'afb9dde57625409e895f28ac5b426c6d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:48.441065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 05 16:48:48.443108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:48.445339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 242/478] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Sep 5 16:48:48 2022] POST /identity/v3/roles => generated 144 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:48.445473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:48.491292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:48.545919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6051ea3-f4f7-43d5-9eea-3d8b0e941880 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:48.581450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78413da7-3db7-4094-9320-008208b32e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:48.582140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78413da7-3db7-4094-9320-008208b32e37 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:48.582441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78413da7-3db7-4094-9320-008208b32e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:48.582682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78413da7-3db7-4094-9320-008208b32e37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:48.593341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78413da7-3db7-4094-9320-008208b32e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-78413da7-3db7-4094-9320-008208b32e37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:48.598912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:48.601031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:48.603064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 237/479] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:48:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:48.609531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:48.714335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4fd28-7b6b-45a3-a22d-ecd75475e091 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:48.750379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe85f2-6ae8-4a85-ab26-87f9d4c257f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:48.751268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe85f2-6ae8-4a85-ab26-87f9d4c257f7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:48.751719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe85f2-6ae8-4a85-ab26-87f9d4c257f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:48.752133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe85f2-6ae8-4a85-ab26-87f9d4c257f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:48.752974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe85f2-6ae8-4a85-ab26-87f9d4c257f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:48.753446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe85f2-6ae8-4a85-ab26-87f9d4c257f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:48.755359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe85f2-6ae8-4a85-ab26-87f9d4c257f7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:48.765698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 243/480] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:49.750145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82473a-6acf-4c6f-8527-d2b21b3f92c4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:49.751145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82473a-6acf-4c6f-8527-d2b21b3f92c4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:49.751626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82473a-6acf-4c6f-8527-d2b21b3f92c4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:49.753458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 238/481] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:49.758660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823f22c-f248-4fc1-902a-408a90ccd71b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:49.759486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823f22c-f248-4fc1-902a-408a90ccd71b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:49.759968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823f22c-f248-4fc1-902a-408a90ccd71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:49.778946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2823f22c-f248-4fc1-902a-408a90ccd71b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:49.784550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2823f22c-f248-4fc1-902a-408a90ccd71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:49.787104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:49.861108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 244/482] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:49.933293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05fd8-86a1-406c-9835-87fc04ae17a0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:49.933574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05fd8-86a1-406c-9835-87fc04ae17a0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:49.933851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b05fd8-86a1-406c-9835-87fc04ae17a0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:49.935530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 239/483] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:49.941592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2e98-7bb9-4bfb-ab00-29dee0c0166c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:49.941839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2e98-7bb9-4bfb-ab00-29dee0c0166c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:49.942110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a2e98-7bb9-4bfb-ab00-29dee0c0166c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:49.960505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540b6fc-ad3a-4086-82f0-93d5a1839008 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:49.961008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540b6fc-ad3a-4086-82f0-93d5a1839008 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:49.961287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540b6fc-ad3a-4086-82f0-93d5a1839008 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:49.961614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b40a2e98-7bb9-4bfb-ab00-29dee0c0166c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:49.962838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b40a2e98-7bb9-4bfb-ab00-29dee0c0166c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:49.962942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 240/484] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:49.967356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8574a9-7edb-4dc6-bac9-4ca373b80c49 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:49.967632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8574a9-7edb-4dc6-bac9-4ca373b80c49 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:49.967901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8574a9-7edb-4dc6-bac9-4ca373b80c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:49.996507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bb8574a9-7edb-4dc6-bac9-4ca373b80c49 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:49.997686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bb8574a9-7edb-4dc6-bac9-4ca373b80c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:50.089058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 245/485] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:50.119489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 241/486] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:50.180041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2e98-7bb9-4bfb-ab00-29dee0c0166c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.262997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:50.304951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a2e98-7bb9-4bfb-ab00-29dee0c0166c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:50.349763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:50.350342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:50.350619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:50.350887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:50.365783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:50.383738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:50.384652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:50.385837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 246/487] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:50.392551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8574a9-7edb-4dc6-bac9-4ca373b80c49 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.400124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.438587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:50.502512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8574a9-7edb-4dc6-bac9-4ca373b80c49 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:50.535588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5dbdb-bf2e-4b8f-a7ae-2c93e4650ec2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:50.571480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:50.575596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:50.576261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:50.577175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:50.577454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:50.585514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:50.586180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:50.586457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:50.586722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:50.587385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:50.587686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:50.589142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:50.600988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:50.607247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:50.608100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:50.608921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:50.610341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 242/488] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:50.611287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 247/489] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:50.616314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.617560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.742975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2b567-32e5-4993-bd35-cf2f5162fc4d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:50.743887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4297-20c0-408b-8896-e1f33e8e6311 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:50.746442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:50.780506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:50.781194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:50.781374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:50.781641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:50.782250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e468408-560e-4b85-a192-e724483605be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:50.782640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:50.782640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:50.783489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e468408-560e-4b85-a192-e724483605be None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:50.783910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e468408-560e-4b85-a192-e724483605be None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:50.784019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:50.784494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e468408-560e-4b85-a192-e724483605be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:50.796197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e468408-560e-4b85-a192-e724483605be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:50.797331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 243/490] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:50.798150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e468408-560e-4b85-a192-e724483605be None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:50.798301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4e468408-560e-4b85-a192-e724483605be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:50.814742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:50.815913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:50.816974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:50.816974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:50.816974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:50.818831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 248/491] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:50.819844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e468408-560e-4b85-a192-e724483605be None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.865845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:50.904782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94976e8f-dd93-42ef-bcc5-462bb23abe72 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:50.929310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e468408-560e-4b85-a192-e724483605be None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:50.940330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:50.940940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:50.941304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:50.941612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:50.951854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:50.957314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:50.958369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:50.959193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 244/492] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:50.959837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:50.971376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a9aeb-d7c3-4cc4-b53e-09214b0c853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:50.972460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a9aeb-d7c3-4cc4-b53e-09214b0c853c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:50.972967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a9aeb-d7c3-4cc4-b53e-09214b0c853c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:50.973501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a9aeb-d7c3-4cc4-b53e-09214b0c853c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:50.985223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a9aeb-d7c3-4cc4-b53e-09214b0c853c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:50.986670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a9aeb-d7c3-4cc4-b53e-09214b0c853c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:50.987993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2a0a9aeb-d7c3-4cc4-b53e-09214b0c853c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:51.003310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:51.004412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:51.005558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:51.005558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:51.005558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:51.005558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 249/493] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:51.005956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74fff7-29b3-49cb-be21-052f215949a3 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.006246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74fff7-29b3-49cb-be21-052f215949a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.006521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74fff7-29b3-49cb-be21-052f215949a3 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.011932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 250/494] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:51.012526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74fff7-29b3-49cb-be21-052f215949a3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.073259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf0af9-e479-410a-b434-b6685449f051 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.108118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.108330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.108929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.109211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.109621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.110127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:51.110422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.111812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.120083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74fff7-29b3-49cb-be21-052f215949a3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.128136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 245/495] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:51.128136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.158602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69506ad-4fbc-4474-ad01-3769210060bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.159216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69506ad-4fbc-4474-ad01-3769210060bb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.159591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69506ad-4fbc-4474-ad01-3769210060bb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.160078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69506ad-4fbc-4474-ad01-3769210060bb None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.160618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69506ad-4fbc-4474-ad01-3769210060bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:51.160925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69506ad-4fbc-4474-ad01-3769210060bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.162288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69506ad-4fbc-4474-ad01-3769210060bb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.170759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 251/496] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:51.171795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed51d7a-f870-447a-8fd5-54ad4e7e6db9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.172083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed51d7a-f870-447a-8fd5-54ad4e7e6db9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.172389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed51d7a-f870-447a-8fd5-54ad4e7e6db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.191312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eed51d7a-f870-447a-8fd5-54ad4e7e6db9 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:51.192345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eed51d7a-f870-447a-8fd5-54ad4e7e6db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:51.221048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.237699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1436a815-a928-4def-816d-9db115bcbc5c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.261164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 252/497] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:51.262003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed51d7a-f870-447a-8fd5-54ad4e7e6db9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.284636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.285306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.285586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.285855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.286492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:51.286793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.288225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.304687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 246/498] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:51.304687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.349326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.367384 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed51d7a-f870-447a-8fd5-54ad4e7e6db9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.408386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a35ee-ee6b-46b0-9027-9ad440470579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.409258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a35ee-ee6b-46b0-9027-9ad440470579 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.409679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b51169-b753-4cc3-b95b-d163242c92f6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.409836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a35ee-ee6b-46b0-9027-9ad440470579 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.410241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a35ee-ee6b-46b0-9027-9ad440470579 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.417515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a35ee-ee6b-46b0-9027-9ad440470579 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.419118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a35ee-ee6b-46b0-9027-9ad440470579 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a75a35ee-ee6b-46b0-9027-9ad440470579 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:51.432073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:51.433203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:48:51.434332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:51.436087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 253/499] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:51.437404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee0d84-a950-4c6c-b4e1-6b460b1b0d1d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.437679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee0d84-a950-4c6c-b4e1-6b460b1b0d1d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.437951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee0d84-a950-4c6c-b4e1-6b460b1b0d1d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.439337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 254/500] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:51.440971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee0d84-a950-4c6c-b4e1-6b460b1b0d1d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.450552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.451194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.451478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.451749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.463713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.465701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.468892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:51.482138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:51.483050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:51.483903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:51.483903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:51.483903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:51.483903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 247/501] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:51.484613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.548768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee0d84-a950-4c6c-b4e1-6b460b1b0d1d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.583158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a84de-4322-4f6c-bc8e-a06a81011447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.583784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a84de-4322-4f6c-bc8e-a06a81011447 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.584065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a84de-4322-4f6c-bc8e-a06a81011447 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.584365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a84de-4322-4f6c-bc8e-a06a81011447 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.585432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b9634-3df2-4de6-b57d-90713c45c74e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.591505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a84de-4322-4f6c-bc8e-a06a81011447 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.592970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a84de-4322-4f6c-bc8e-a06a81011447 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d25a84de-4322-4f6c-bc8e-a06a81011447 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:51.604675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:51.606023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:51.607235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 255/502] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:51.607874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9919-d3db-4f11-b5fc-64fa6884b8db None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.607874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9919-d3db-4f11-b5fc-64fa6884b8db None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.608766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b9919-d3db-4f11-b5fc-64fa6884b8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.616788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.622190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.622802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.623081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.623349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.624123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:51.624407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.625842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.627514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a22b9919-d3db-4f11-b5fc-64fa6884b8db None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:51.628953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a22b9919-d3db-4f11-b5fc-64fa6884b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:51.661598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 248/503] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:51.662464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.697901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 256/504] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:51.698733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b9919-d3db-4f11-b5fc-64fa6884b8db None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.724377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.771956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afac1-8db7-44ef-b63e-4db51f581679 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.799966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b9919-d3db-4f11-b5fc-64fa6884b8db None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.808526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.809707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.810127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.810667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.811313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:51.811618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.813055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:51.829673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 249/505] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:51.829673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.846146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.846146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.846146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.846146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.847279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.858779 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:51.864086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:51.864980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 257/506] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:51.866395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:51.947612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf9360-a00d-485e-9fd9-751ed8d60dae None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.962107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:51.976987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80724d-d2ad-40d1-9569-05b951a82724 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:51.984231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:51.984834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:51.985121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:51.985419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:51.986110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:51.986414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:51.987979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.004765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 250/507] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:52.005594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.012602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.013206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.013481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.013746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.077509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.085728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.087260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.109685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e23ae8-d771-498e-8317-84a230f40a11 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.146144 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.146773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.147046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.147311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.154521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.155976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:52.169061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:52.170029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 251/508] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:52.170901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.199098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 258/509] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:52.199875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.206868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.279267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1071b9b-f9af-4ef9-83bd-e0bec03d584a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.296036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.302946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2a566-f3ca-40a2-893b-52bf79860426 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.315413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.316041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.316888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.317176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.317832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:52.318135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.319932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.328261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 252/510] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:52.329056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.338146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.338819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.339094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.339367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.350893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.352448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:52.368740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:52.369678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:52.370508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:52.370508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:52.370508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:52.370508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 259/511] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:52.370716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.449813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.457166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f671d0-b1cc-449d-b5b0-17a9800bf806 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.486968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9199aee-d9e6-4d99-bfa4-6d8372184ca0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.492045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.492680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.492959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.493231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.521403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.522001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.522280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.522545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.523245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:52.523547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.525020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.559240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.560492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 260/512] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:52.560794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.561307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.566340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.652000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.660382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 253/513] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:52.661160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd26ea0-a617-4aac-9a85-df8bfeaa63fe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.661316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.698519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.699147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.699426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.699699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.710424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.711829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:52.726456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:52.727520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:52.728882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:52.728882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:52.728882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:52.728882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 261/514] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:52.728882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.764531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69933e-4d0e-496b-8de1-26a51100c202 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.786575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.798140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.798775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.799050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.799319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.809821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.811305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:52.827900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:52.829020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:52.830127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:52.830127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:52.830127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:52.833992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 254/515] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:52.834712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.840478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445ef8-e1d2-4313-a035-99ec165d6818 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.875925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.876579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.876856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.877124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.877763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:52.878069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:52.879968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:52.883223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:52.892899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 262/516] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:52.893813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:52.940305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e727500-9cab-4102-b57f-95d644828d66 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:52.975561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:52.976083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:52.976388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:52.976659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:52.997224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bd925-dc72-4713-aaa7-ee7bd986818c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.014032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.032367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.033093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.033374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.033647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.045134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.046631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.050147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.051732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:53.062149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:53.063049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:53.063894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:53.063894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:53.063894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:53.063894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 263/517] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:53.064456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.100215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.154192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 255/518] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:53.155024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.170616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1539e5b-f0f8-470f-8073-3c57b980c9e1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.204865 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.205574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.205852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.206122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.206759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:53.207058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.208591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.221089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 264/519] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:53.221953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.250892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.263038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de64f7-f719-4bed-9883-5a229c6dccfc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.295885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.296522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.296798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.297068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.297861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:53.298222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.300038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.314506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 256/520] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:53.315562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.324864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.325661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31771ac-a106-4776-bed7-e02169b92742 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.360120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.360756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.361034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.361301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.361934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.363590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.385871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 265/521] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:48:53 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:48:53.386759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.432424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16412c-a96d-4947-9bff-ceb995662efa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.473611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.474343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.474808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.474914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.475521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:53.475827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.477316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.483438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.492132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 257/522] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:53.492877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.510846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa806b-4fb0-43a7-b79a-8db8f4b1d92b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.549790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.550498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.550808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.551076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.562640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.564869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.571358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:53.581629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:53.582591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:53.583601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:53.583601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:53.583601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:53.583601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 266/523] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:53.584213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.612132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b37d58-8501-4892-be4a-ea8e7bb87b91 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.650466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.651101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.651376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.651645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.658805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.660619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:48:53.673513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:53.674406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 258/524] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:53.675473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.697273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697545-0ce6-49b6-bbb2-be0151537e3b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.733492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.734118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.734391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.734648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.737385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.743581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.745058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:53.756301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:53.757228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 267/525] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:53.758200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.787568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a70ef-a557-4612-a7bb-e3efda903f81 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.814857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:53.825884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.826515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.826788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.827056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.834103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.835571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:48:53.846925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:53.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 259/526] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:53.854401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.876614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71ad87-03c1-40ca-905a-4e986b1cedf4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.911051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:53.911751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:53.912054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:53.912419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:53.913057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:53.913354 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:53.914721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:53.930908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 268/527] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:53.930908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:53.965640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9c1f9-96f5-463c-98ec-038908393223 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:53.993116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.003612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.004440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.004768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.005050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.005797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:54.006043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.007496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.030736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 260/528] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:54.030736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.042981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b6b7c-9934-4b57-a234-a9a00cba5f10 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.078114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.080257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.080877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.081198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.081462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.082412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:54.082714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.084158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.106696 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 269/529] 10.222.0.23 () {56 vars in 1139 bytes} [Mon Sep 5 16:48:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:54.106696 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.141823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cac7c-cae0-46cf-9e0c-52de60a37eec None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.179907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.180542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.180823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.181093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.181781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:54.182083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.183534 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.196080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 261/530] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:54.201338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.204205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a32eb-d25b-4107-b586-2ed44cb30d28 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.219059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.245448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.246082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.246517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.246799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.255354 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.257336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:48:54.269965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:54.271071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:54.271910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 270/531] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:54.273008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.304228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549681b3-2a57-41f8-b6e9-cbedb2f7e787 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.313378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.343289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.343897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.344194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.344470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.345180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=c30e20ce4d63452095aedb60249f0cce) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:54.345483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.346918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.352926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 262/532] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/role_assignments?user.id=c30e20ce4d63452095aedb60249f0cce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:54.357525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.383877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16099f1b-8a7d-42c7-9630-016338f94c1b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.424648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.425408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.425838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.426177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.476770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.490588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ca77-2c86-41d3-bdb5-8e12e6a8cbaf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.509468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.510939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.531737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.532380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.532656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.532929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.533644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=36c795d7c6bc4c519c15f32489c44583) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:54.534408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.536238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.544002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 263/533] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/role_assignments?user.id=36c795d7c6bc4c519c15f32489c44583&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:54.545984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.617731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.698054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 271/534] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:54.704105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.740555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354db7a8-4b61-4a85-8837-7f5a0465252b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.797603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:54.798227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:54.798502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:54.798770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:54.799467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:54.799769 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:54.801654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:54.824556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.831487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 264/535] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:54.837968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:54.896913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879c739-e526-451b-8da2-0c60010d02a9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.951170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:54.960588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71b3b8-99fe-4118-b764-e316b0ed01e3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:54.999329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:55.000471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:55.000962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:55.001420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:55.002327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=05a016553950460dbb736ced2cae4f62) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:55.002823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:55.004757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:55.011771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 265/536] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/role_assignments?user.id=05a016553950460dbb736ced2cae4f62&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:55.041717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:55.042385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:55.042665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:55.042931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:55.066565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:55.068038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:55.090411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:55.091372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:55.092258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:55.092258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:55.092258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:55.092258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 272/537] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:55.097485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:55.244190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fd7f2-0e34-4a1e-869e-6c776534fd91 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:55.284668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7529cb4-46c1-4c17-b355-77b67ca3d6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:55.285397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7529cb4-46c1-4c17-b355-77b67ca3d6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:55.285672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7529cb4-46c1-4c17-b355-77b67ca3d6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:55.285952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7529cb4-46c1-4c17-b355-77b67ca3d6c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:55.286599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7529cb4-46c1-4c17-b355-77b67ca3d6c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:55.286897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7529cb4-46c1-4c17-b355-77b67ca3d6c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:55.288387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7529cb4-46c1-4c17-b355-77b67ca3d6c1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:55.312130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 266/538] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:55.329740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:55.461803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c74bd-760e-44f5-8c70-bc49faaee72f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:55.462413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c74bd-760e-44f5-8c70-bc49faaee72f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:55.462883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c74bd-760e-44f5-8c70-bc49faaee72f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:55.464692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 267/539] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:55.476774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee22d6-3b71-43be-bd6f-70b6c562593d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:55.477261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee22d6-3b71-43be-bd6f-70b6c562593d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:55.477672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee22d6-3b71-43be-bd6f-70b6c562593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:55.503343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a946b-e15a-4530-8a12-23ea6f648974 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:55.520471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b0ee22d6-3b71-43be-bd6f-70b6c562593d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:55.528849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b0ee22d6-3b71-43be-bd6f-70b6c562593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:55.529744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:55.540244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:55.540883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:55.541160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:55.541422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:55.549007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:55.550611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:48:55.568337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:55.569244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:55.570087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 273/540] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:55.571056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:55.572321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:55.717073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 268/541] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:55.772267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0804da-496b-4331-a995-669b16f720ba None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:55.855226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:55.856210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:55.856485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:55.856750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:55.857449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:55.857747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:55.859172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:55.878825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 274/542] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:48:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:55.893290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee22d6-3b71-43be-bd6f-70b6c562593d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:56.103978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee22d6-3b71-43be-bd6f-70b6c562593d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:56.153172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:56.161795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:56.162422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:56.162698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:56.162966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:56.163707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=fcbcd7638cc84cdd87a1eb98f5926425) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:56.164013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:56.165518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:56.180599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 269/543] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:55 2022] GET /identity/v3/role_assignments?user.id=fcbcd7638cc84cdd87a1eb98f5926425&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:56.240542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:56.337770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83d88f-5cfc-48ba-b2eb-d2020a0187cf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:56.385432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:56.386086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:56.386362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:56.386698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:56.398542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:56.403521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:56.404654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:56.405717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 275/544] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:56.413341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:56.436727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:56.542914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c21bdc-2128-4c2d-bf21-fab2940277b9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:56.591549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12744d33-9020-4b3f-a816-5567aa61ce7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:56.592199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12744d33-9020-4b3f-a816-5567aa61ce7b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:56.592483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12744d33-9020-4b3f-a816-5567aa61ce7b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:56.592755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12744d33-9020-4b3f-a816-5567aa61ce7b None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:56.593415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12744d33-9020-4b3f-a816-5567aa61ce7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:56.593714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12744d33-9020-4b3f-a816-5567aa61ce7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:56.595101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12744d33-9020-4b3f-a816-5567aa61ce7b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:56.604044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 270/545] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:56.610138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:56.698692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1f964-05fb-4c5c-899d-9b69d729e804 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:56.698898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1f964-05fb-4c5c-899d-9b69d729e804 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:56.699180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1f964-05fb-4c5c-899d-9b69d729e804 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:56.700830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 271/546] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:48:56.705812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9604993-29b1-4b83-965f-b34e6b29c118 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:56.706072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9604993-29b1-4b83-965f-b34e6b29c118 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:56.706376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9604993-29b1-4b83-965f-b34e6b29c118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:56.729351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d9604993-29b1-4b83-965f-b34e6b29c118 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:56.730869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d9604993-29b1-4b83-965f-b34e6b29c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:56.766077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510d89-d008-4aaa-8b7b-34dafdd6af92 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:56.775666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:56.807898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:56.808555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:56.808829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:56.809092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:56.821412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:56.822874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:56.839334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:56.840290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:56.841125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:56.841125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:56.841125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:56.845132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 276/547] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:56.847515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:56.904460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 272/548] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:56.984198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371fd49-e9d0-4dd9-9a88-83ceb0022802 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:57.039627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a824f-4241-43ea-97bc-808814aae12d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.039972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a824f-4241-43ea-97bc-808814aae12d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.040299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a824f-4241-43ea-97bc-808814aae12d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.040605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:57.041219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.041491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.041758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.041920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 273/549] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:57.042458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:57.042759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:57.044200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:57.046796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f028fb-9249-4bfc-9c38-4bf43f932923 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.047083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f028fb-9249-4bfc-9c38-4bf43f932923 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.047358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f028fb-9249-4bfc-9c38-4bf43f932923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.063255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 277/550] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:57.066707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:57.068385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.085531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b6f028fb-9249-4bfc-9c38-4bf43f932923 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:57.086728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b6f028fb-9249-4bfc-9c38-4bf43f932923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:57.089940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:57.181557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea49e7e-8428-4654-a691-068d308dfb9c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:57.220761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:57.221403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.221675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.221943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.231210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:57.233410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:57.237155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 274/551] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 05 16:48:57.238084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f028fb-9249-4bfc-9c38-4bf43f932923 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:57.246874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:57.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:48:57.250887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:57.260177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 278/552] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:48:57.260177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.363836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:57.415541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23504c68-f946-40d6-ba20-9305883fabd1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:57.441614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f028fb-9249-4bfc-9c38-4bf43f932923 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:57.454823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116b9f5-1493-4ae9-992d-75d8c6e97739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:57.455431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116b9f5-1493-4ae9-992d-75d8c6e97739 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.455792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116b9f5-1493-4ae9-992d-75d8c6e97739 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.456201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116b9f5-1493-4ae9-992d-75d8c6e97739 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.457038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116b9f5-1493-4ae9-992d-75d8c6e97739 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:57.457337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116b9f5-1493-4ae9-992d-75d8c6e97739 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:57.458765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116b9f5-1493-4ae9-992d-75d8c6e97739 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:57.481272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 279/553] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:48:57.481272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb69ba-3f98-45f9-9347-4b8b6976f643 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.481272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb69ba-3f98-45f9-9347-4b8b6976f643 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.481272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb69ba-3f98-45f9-9347-4b8b6976f643 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.484376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 280/554] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:57.485359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb69ba-3f98-45f9-9347-4b8b6976f643 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e000c3c-b9dc-49e7-9c92-757a3edaad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:57.487239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e000c3c-b9dc-49e7-9c92-757a3edaad20 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.487746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e000c3c-b9dc-49e7-9c92-757a3edaad20 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.488363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e000c3c-b9dc-49e7-9c92-757a3edaad20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.490833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:57.501936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e000c3c-b9dc-49e7-9c92-757a3edaad20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8e000c3c-b9dc-49e7-9c92-757a3edaad20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:57.507267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:57.508339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:57.509158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:57.512928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 275/555] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:57.514344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc29b-ce62-4397-a96e-025d8bf6bf43 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.514817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc29b-ce62-4397-a96e-025d8bf6bf43 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.515228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc29b-ce62-4397-a96e-025d8bf6bf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.535669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-59dfc29b-ce62-4397-a96e-025d8bf6bf43 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:57.537316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-59dfc29b-ce62-4397-a96e-025d8bf6bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:57.606049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb69ba-3f98-45f9-9347-4b8b6976f643 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:57.616326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 276/556] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:57.616326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfc29b-ce62-4397-a96e-025d8bf6bf43 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.663657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:57.664301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.664626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.664890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.669620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:57.739739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfc29b-ce62-4397-a96e-025d8bf6bf43 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:57.753592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:57.755654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:57.765131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:57.777433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:57.778192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.778568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.778931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.779728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:57.780121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:57.781615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:57.791245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 277/557] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:57.792410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.880367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 281/558] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:48:57.881465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.910397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f0f41-ad41-44e9-bfe8-95d6359f39ad None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:57.919318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:57.946970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:57.947610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:57.947893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:57.948182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:57.959155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:57.964010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:57.964904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:57.965706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 278/559] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:57.967493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:57.992998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeb14b-4e5a-4ef7-8e1e-5b1005418159 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.016511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.037891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.038529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.039133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.039133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.049549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.051058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:58.065985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:58.066925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:58.067783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:58.067783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.067783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:58.067783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 282/560] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:58.068000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.080424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb90270-1357-45c8-938b-525173f7f9a1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.117134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.117777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.118059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.118335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.128689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.130157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:58.145078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:58.145996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:58.146857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:58.146857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.146857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:58.146857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 279/561] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:58.147071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.173386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.188507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3adb3-2a9c-47fc-8715-03a8560a6c88 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.223814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.224490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.224771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.225044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.235662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:58.241008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:58.241914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:58.242803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 283/562] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:58.243524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.257804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.261953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d77f-be5f-4316-a0b9-f91574f0eb0c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.299553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.300367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.300649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.300926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.301771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:58.302077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.303456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.311550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 280/563] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:58.312416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.349227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d8a6f-597b-4c61-a9c0-39349b265cdd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.386452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.387311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.387596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.387870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.388546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:58.388851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.390224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.402177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 284/564] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:58.402985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.415168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.417370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd1390-2104-4be3-ba2f-0ef9d4cfc0f4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.451265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.451852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.452129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.452432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.453063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:58.453356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.454747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.466372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 281/565] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:58.467152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.488731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.511134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42345cf-271f-471f-9329-72253a621f71 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.545676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57d2c8-96f1-4eed-806b-259acd191bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.546289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d2c8-96f1-4eed-806b-259acd191bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.546587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d2c8-96f1-4eed-806b-259acd191bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.546851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d2c8-96f1-4eed-806b-259acd191bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.547512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d2c8-96f1-4eed-806b-259acd191bf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:58.547818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d2c8-96f1-4eed-806b-259acd191bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.549306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d2c8-96f1-4eed-806b-259acd191bf9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.557329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 285/566] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:58.558477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489354e0-db14-4eec-baea-a40b72fcd8c4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.558764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489354e0-db14-4eec-baea-a40b72fcd8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.559038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489354e0-db14-4eec-baea-a40b72fcd8c4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.560245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 286/567] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:48:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:48:58.560937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489354e0-db14-4eec-baea-a40b72fcd8c4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.571956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f903b6c-accb-49d9-b401-847c78f54337 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.605971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.606609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.606889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.607149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.617191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.618664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.630196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:58.632901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:58.634006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:58.634883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:58.634883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.634883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:58.634883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 282/568] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:58.634883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.663662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489354e0-db14-4eec-baea-a40b72fcd8c4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.698148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.698781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.699030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.699324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.705548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.706661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.708337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:58.719745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:58.720951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:48:58.722037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:58.725819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 287/569] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:58.726575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.736642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855951b1-424a-47e4-bb62-81ba1f23b86b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.770017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470626-9545-49a1-af3b-c3173d048ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.770577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470626-9545-49a1-af3b-c3173d048ecb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.770877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470626-9545-49a1-af3b-c3173d048ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.771121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470626-9545-49a1-af3b-c3173d048ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.777758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470626-9545-49a1-af3b-c3173d048ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.779244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470626-9545-49a1-af3b-c3173d048ecb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-eb470626-9545-49a1-af3b-c3173d048ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:48:58.790415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:58.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:58.792264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 283/570] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:58.793315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b26cb9-2352-4a1a-94df-ca05b54657ec None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.793424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b26cb9-2352-4a1a-94df-ca05b54657ec None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.793610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b26cb9-2352-4a1a-94df-ca05b54657ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.811287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d8b26cb9-2352-4a1a-94df-ca05b54657ec None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:48:58.812467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d8b26cb9-2352-4a1a-94df-ca05b54657ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:48:58.830414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02db1d-2928-4d36-8ea3-1f978e73e757 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:58.860508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.867211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:58.867843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:58.868117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:58.868422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:58.880080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:58.880330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 284/571] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:48:58.881179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b26cb9-2352-4a1a-94df-ca05b54657ec None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.881643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:58.897243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:58.898302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:58.899314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:58.899314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:58.899314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:58.901098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 288/572] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:58.902167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:58.955753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:58.998578 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b26cb9-2352-4a1a-94df-ca05b54657ec None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.012829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945ca26-221f-4a3b-adde-5f70250f4df4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.034359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.035051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.035370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.035582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.036258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.036553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.037927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.049628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 285/573] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.049778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.050468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.050828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.051267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.051537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.052289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.052589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.053992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.072422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 289/574] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:48:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.073318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.119641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:59.162773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47e8f-e410-4b10-84f6-1cab84d6e4ab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.178729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1089d2-66f2-43f1-9b04-fa9cc9c1619e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.184259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:59.195579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.196214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.196495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.196767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.197450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.197753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.199188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.211467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 286/575] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.212290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.212397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.213004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.213276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.213546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.214179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.214479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.215832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.227465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 290/576] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.228360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.315021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab638925-d2f6-47bc-a07f-1b0e42049a3e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.324902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:59.327712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc7146-9044-4066-a226-916e9315e148 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.350121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.350760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.351040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.351311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.358642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.360130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.362758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.363812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.364092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.364824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:59.371818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:59.372807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:48:59.373744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:59.377659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 287/577] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:59.378591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.378870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.380508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:48:59.396579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:59.402438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:48:59.403519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:48:59.403519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:48:59.403519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:48:59.403519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 291/578] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:59.403519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.428719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:59.484901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda55bf-a1a4-4d10-8eaf-799a8fdb4782 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.504037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8418d13-913f-4cac-8318-34ca3fa5b24a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.519306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.520286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.520573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.520848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.530980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:48:59.536779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:59.537677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:59.538591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 288/579] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:59.539276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.545388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.546024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.546298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.546564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.547282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2756036297064cc19d5fd5d1861703c3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.547615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.549071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.555129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 292/580] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/role_assignments?user.id=2756036297064cc19d5fd5d1861703c3&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.555890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.576262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:59.660690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac3673-a2d3-451d-9f7b-f11e3f2f3a8e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.662410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:59.673290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed66e-6add-4ad6-afa1-18014b85b612 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.699087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.699726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.700003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.700306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.709576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.711056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.713784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.714398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.714673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.714940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.715665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.715965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.717431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:48:59.723788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:48:59.724728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:48:59.725670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 289/581] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:48:59.726246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.745967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 293/582] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.745967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.819568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:48:59.841170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba204d8-208f-4f49-9c2b-eff1737d793d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.847589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c327d31-c2a0-40ed-a161-1de6ec4f9724 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:48:59.877106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.877728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.878008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.878649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.878969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.879391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.880818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.885110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:48:59.885724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:48:59.886000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:48:59.886266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:48:59.886918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:48:59.887219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:48:59.889192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:48:59.893223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 290/583] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.894530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.899832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 294/584] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:48:59.900905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:48:59.924680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.008338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01478ae-556c-4822-9d30-7430852dd80b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.010856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee10f-2d84-4ef6-ac42-3b810d7a2ab9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.043983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.044634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.044915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.045078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.045220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.045701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.045899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:00.045995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.046207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.046301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.047646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.048926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.060316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.061771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.070947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 291/585] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:00.071677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:00.076608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:00.077525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:00.078457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:00.078457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:00.078457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:00.082415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 295/586] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:00.083107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.155679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.177695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00e8ad-a75d-4f8a-bd29-fed8ad3878ac None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.183328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729a4cf-5ce3-4cf5-a037-15c65d7529b2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.212375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.212997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.213274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.213551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.217443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.218072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.218349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.218621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.220220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.221659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.229419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.231164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:00.233220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:00.234072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:00.234959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 292/587] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:00.235465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:00.252562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:00.253774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:00.254665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:00.254665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:00.254665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:00.254665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 296/588] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:00.254665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.301299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.346193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298194c-d777-4c26-8c8d-31ff4402f37c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.365194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebf0b2-8369-4473-8d84-07a1b1a52ec2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.383242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.383875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.384178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.384448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.395748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.398319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.400709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.404337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.404949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.405227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.405499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.406190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:00.406442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.407807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:00.417386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:00.418302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:00.424716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:00.424716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:00.424716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:00.424716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 293/589] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:00.424716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.425001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 297/590] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:00.425001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.524474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.529023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c3068-92ea-4457-bf3f-917346cad01c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.533968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c82441-9adb-445b-a7b6-efc78585e0f3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.568086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.568666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.568784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.568992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.569263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.569368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.569532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.569803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.569895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:00.570194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.570482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:00.570785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.571623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.572214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.589158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 294/591] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:00.589158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.589448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 298/592] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:00.589448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.647990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.698091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d9cbb-1215-4dfe-b6ac-edf2cad05515 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.703600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2dba1-7d80-46c1-a382-6df5c70c595c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.734581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.735196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.735477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.735750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.736413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:00.736718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.737330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.737987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.738186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.738322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.738501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.745278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.746701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.754441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 295/593] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:00.754441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:00.758373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:00.759207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 299/594] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:00.760710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.788529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.871464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c68779-b8d3-4d05-aadd-562a4a66325e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.872004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f10d0-a7b4-47dc-bbf8-5f9d8d66dbd1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:00.887976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:00.908631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.909296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.909779 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.910060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/c30e20ce4d63452095aedb60249f0cce/roles/ba985e88b6d44ba38445fe86872360b2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.912420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:00.914044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:00.914478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:00.914915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:00.923342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.924507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=c30e20ce4d63452095aedb60249f0cce, role_id=ba985e88b6d44ba38445fe86872360b2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:00.925259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.925998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:00.938032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:00.939152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 16:49:00.940202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:00.940728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 300/595] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:00 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/c30e20ce4d63452095aedb60249f0cce/roles/ba985e88b6d44ba38445fe86872360b2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:00.940728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:00.942160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 296/596] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:00.943103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:01.028405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:01.077288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec3007-f071-48c4-a23f-212434d4196c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.080857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a611db-ee1a-432c-bd60-bb1041cb2881 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.122101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.122756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.123032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.123304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.132515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.133976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.134481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.135262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.135942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.136432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.137700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:01.138012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.139463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:01.147657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:01.148578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 301/597] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:01.150104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:01.155970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 297/598] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:01.160390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:01.195843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:01.274081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81264c-8ca9-463c-bdc0-729fc42a79e1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.278402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e5c1d-3a0d-4c2b-bbaa-1397929bd4a6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.314283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.314979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.315352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.315762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:01.316430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.318248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:01.318841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.321335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.321659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.322556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.323189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.323457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.324216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:01.324813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.326815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.337039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 298/599] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:01.338249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:01.348597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 302/600] 10.222.0.23 () {56 vars in 1139 bytes} [Mon Sep 5 16:49:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:01.349597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:01.448365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:01.458968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14237697-a382-42c5-b958-19152acf54d7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.466269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0daa5-06c6-4392-a430-c7e69727f79a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.499999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0b4b7-0456-4568-9152-c6391d348caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.500697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0b4b7-0456-4568-9152-c6391d348caf None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.500977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0b4b7-0456-4568-9152-c6391d348caf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.501247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0b4b7-0456-4568-9152-c6391d348caf None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/36c795d7c6bc4c519c15f32489c44583/roles/ba985e88b6d44ba38445fe86872360b2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.501697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.502423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.502712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.502981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/05a016553950460dbb736ced2cae4f62/roles/ba985e88b6d44ba38445fe86872360b2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.517319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=05a016553950460dbb736ced2cae4f62, role_id=ba985e88b6d44ba38445fe86872360b2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.517603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0b4b7-0456-4568-9152-c6391d348caf None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=36c795d7c6bc4c519c15f32489c44583, role_id=ba985e88b6d44ba38445fe86872360b2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.518745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.519048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0b4b7-0456-4568-9152-c6391d348caf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.533177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 303/601] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:01 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/05a016553950460dbb736ced2cae4f62/roles/ba985e88b6d44ba38445fe86872360b2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:01.533997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:01.545933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 299/602] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:01 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/36c795d7c6bc4c519c15f32489c44583/roles/ba985e88b6d44ba38445fe86872360b2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:01.545933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335d54-6fdc-45d9-a10b-5d8b61103e6d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.546382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335d54-6fdc-45d9-a10b-5d8b61103e6d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.546973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4335d54-6fdc-45d9-a10b-5d8b61103e6d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.547970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 300/603] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:01.561696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c22ee-baae-48ba-b8ac-5858e494bc6c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.562044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c22ee-baae-48ba-b8ac-5858e494bc6c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.562235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c22ee-baae-48ba-b8ac-5858e494bc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.601662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:01.620417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a20c22ee-baae-48ba-b8ac-5858e494bc6c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:01.621638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a20c22ee-baae-48ba-b8ac-5858e494bc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:01.677092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec7574-1594-4496-b03f-31637b1e9603 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.719405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.720076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.720368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.720640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.733435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.734985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:01.751996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:01.753125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:01.754068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:01.754068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:01.754068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:01.754068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 304/604] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:01.759399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:01.771500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 301/605] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:01.860007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:01.870657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f2a3f-a626-419a-826c-9abf6846ba7d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:01.910388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:01.911173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:01.911449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:01.911720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:01.912383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:01.912684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:01.914050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:01.926574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 305/606] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:01.934009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c22ee-baae-48ba-b8ac-5858e494bc6c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:02.031992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:02.073014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c22ee-baae-48ba-b8ac-5858e494bc6c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:02.119791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:02.120435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:02.120641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:02.120913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:02.130631 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:02.132089 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:02.147546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:02.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:02.149394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 302/607] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:02.155021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:02.166064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:02.333625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550a354-c14e-48fc-91c2-9d9597ef9dc3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:02.340530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edea92a-5a18-4e02-8057-0304caccfb7d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:02.378146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:02.385011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:02.385659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:02.385935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:02.386203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:02.397156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:02.403888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:02.404832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:02.405672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 303/608] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:02.413172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:02.414188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:02.414804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:02.415074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:02.415335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:02.416024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:02.416347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:02.417751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:02.439839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 306/609] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:02.457114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:02.482292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:02.529019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210f567-39fd-448f-96e9-7f4349efecac None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:02.563809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:02.564739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:02.565194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:02.565631 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:02.566497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:02.567064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:02.568727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:02.577454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 304/610] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:02.591848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:02.651903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517f957-3b9c-4804-b61f-663290f68205 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:02.696564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:02.696564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:02.696564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:02.696564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/fcbcd7638cc84cdd87a1eb98f5926425/roles/ba985e88b6d44ba38445fe86872360b2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:02.742174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=fcbcd7638cc84cdd87a1eb98f5926425, role_id=ba985e88b6d44ba38445fe86872360b2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:02.743576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:02.766855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219d7c0-fc8a-4661-a0ab-caaf18db326a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:02.772515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:02.785602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 307/611] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:02 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/fcbcd7638cc84cdd87a1eb98f5926425/roles/ba985e88b6d44ba38445fe86872360b2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:02.846630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:02.847336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:02.847619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:02.847894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:02.868922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:02.870537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:02.890539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:02.891661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:02.892679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:02.892679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:02.892679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:02.892679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 305/612] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:02.900128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:02.969407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:03.015517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31934c-c8a7-46e9-b84d-73dec01665cb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:03.053182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:03.053789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.054062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.054332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.054967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:03.055267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:03.056659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:03.072272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 308/613] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:03.074702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:03.177846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6181013-0386-4212-981a-ac63d9386b56 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:03.185109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:03.212718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e110a-c818-4aa5-a86e-03111c97900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:03.213903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e110a-c818-4aa5-a86e-03111c97900e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.214226 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e110a-c818-4aa5-a86e-03111c97900e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.214499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e110a-c818-4aa5-a86e-03111c97900e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.222820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e110a-c818-4aa5-a86e-03111c97900e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:03.224469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e110a-c818-4aa5-a86e-03111c97900e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-992e110a-c818-4aa5-a86e-03111c97900e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:03.236932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:03.237911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:03.238816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:03.243124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 306/614] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:03.244565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:03.297864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188b9ae-4770-4291-8e6f-8f2da2961b7d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.298197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188b9ae-4770-4291-8e6f-8f2da2961b7d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.298422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188b9ae-4770-4291-8e6f-8f2da2961b7d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.300083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 307/615] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:03.305389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24766bd-1553-46f7-814d-e6f15d7aacf7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.305668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24766bd-1553-46f7-814d-e6f15d7aacf7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.305939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24766bd-1553-46f7-814d-e6f15d7aacf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.342317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b24766bd-1553-46f7-814d-e6f15d7aacf7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:03.343521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b24766bd-1553-46f7-814d-e6f15d7aacf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:03.354995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c09658-2635-4484-8fb7-abea626a911e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:03.378301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:03.396879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:03.397500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.397771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.398037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.398766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:03.399063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:03.400513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:03.425539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 309/616] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Sep 5 16:49:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:03.427636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:03.539160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd14b11-be99-4a12-9667-40142efaf1e2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:03.576584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:03.577259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.577536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.577800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.582695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 308/617] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:03.588545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:03.590037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:03.604951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:03.606270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:03.607137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:03.607137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:03.607137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:03.611093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 310/618] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:03.612551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24766bd-1553-46f7-814d-e6f15d7aacf7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:03.708972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:03.716806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24766bd-1553-46f7-814d-e6f15d7aacf7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:03.754143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:03.754753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.755026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.755295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.755930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:03.756256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:03.757644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:03.772452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 309/619] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:03.775369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:03.808209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:03.891400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16795db5-d8ce-415b-aefd-30a2e1404e0d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:03.901837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:03.939598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9bf5b-7973-4b50-94c1-efbe00d00c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:03.940684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9bf5b-7973-4b50-94c1-efbe00d00c62 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.941494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9bf5b-7973-4b50-94c1-efbe00d00c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.942193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9bf5b-7973-4b50-94c1-efbe00d00c62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.949923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9bf5b-7973-4b50-94c1-efbe00d00c62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:03.951626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9bf5b-7973-4b50-94c1-efbe00d00c62 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f0e9bf5b-7973-4b50-94c1-efbe00d00c62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:03.965729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:03.966813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:03.967699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:03.974738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 311/620] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:03.976059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b7fb-5db2-4149-b46d-884fd478a846 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:03.976359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b7fb-5db2-4149-b46d-884fd478a846 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:03.976626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b7fb-5db2-4149-b46d-884fd478a846 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:03.978338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 312/621] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:03.979513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218b7fb-5db2-4149-b46d-884fd478a846 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.036327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4560e4-9a7f-42e0-b5bc-97c62e147283 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.074910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f2284-57e6-433a-b9e9-a4124a812428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.075605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f2284-57e6-433a-b9e9-a4124a812428 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.075889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f2284-57e6-433a-b9e9-a4124a812428 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.076179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f2284-57e6-433a-b9e9-a4124a812428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.080180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:04.089403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f2284-57e6-433a-b9e9-a4124a812428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:04.093733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218b7fb-5db2-4149-b46d-884fd478a846 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2c5f2284-57e6-433a-b9e9-a4124a812428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:04.094699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:04.095595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:04.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 310/622] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:04.097206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee6ae1-0811-42e1-abf6-d90b87fdf661 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.097910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee6ae1-0811-42e1-abf6-d90b87fdf661 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.097910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee6ae1-0811-42e1-abf6-d90b87fdf661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.115749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4bee6ae1-0811-42e1-abf6-d90b87fdf661 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:04.117001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4bee6ae1-0811-42e1-abf6-d90b87fdf661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:04.129110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:04.138010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09184f1-6d7b-4d30-b536-2669de27ff9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.138639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09184f1-6d7b-4d30-b536-2669de27ff9f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.138918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09184f1-6d7b-4d30-b536-2669de27ff9f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.139188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09184f1-6d7b-4d30-b536-2669de27ff9f None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.139897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09184f1-6d7b-4d30-b536-2669de27ff9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:04.140235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09184f1-6d7b-4d30-b536-2669de27ff9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:04.141699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09184f1-6d7b-4d30-b536-2669de27ff9f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:04.154276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 313/623] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:04.155775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e761bc-1a27-4b8f-8ce8-99911a303f96 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.156122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e761bc-1a27-4b8f-8ce8-99911a303f96 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.156431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e761bc-1a27-4b8f-8ce8-99911a303f96 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.160193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 314/624] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:04.160193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e761bc-1a27-4b8f-8ce8-99911a303f96 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.200306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 311/625] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:04.200306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee6ae1-0811-42e1-abf6-d90b87fdf661 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.289900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e761bc-1a27-4b8f-8ce8-99911a303f96 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.316420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee6ae1-0811-42e1-abf6-d90b87fdf661 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.324608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217dd27-e8fc-4545-9604-e35b867de3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.325494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217dd27-e8fc-4545-9604-e35b867de3ac None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.325965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217dd27-e8fc-4545-9604-e35b867de3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.326406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217dd27-e8fc-4545-9604-e35b867de3ac None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.327300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217dd27-e8fc-4545-9604-e35b867de3ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:04.327801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217dd27-e8fc-4545-9604-e35b867de3ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:04.329464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217dd27-e8fc-4545-9604-e35b867de3ac None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:04.338254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 315/626] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:04.339730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf76ab-479f-48d7-9a9e-18dbdc8a0f89 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.340277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf76ab-479f-48d7-9a9e-18dbdc8a0f89 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.340747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf76ab-479f-48d7-9a9e-18dbdc8a0f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.356452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.357146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.357423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.357698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2756036297064cc19d5fd5d1861703c3/roles/ba985e88b6d44ba38445fe86872360b2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.360503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-56cf76ab-479f-48d7-9a9e-18dbdc8a0f89 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:04.362266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-56cf76ab-479f-48d7-9a9e-18dbdc8a0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:04.375293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=2756036297064cc19d5fd5d1861703c3, role_id=ba985e88b6d44ba38445fe86872360b2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:04.376813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:04.381140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:04.389313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:04.397887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 312/627] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:04 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2756036297064cc19d5fd5d1861703c3/roles/ba985e88b6d44ba38445fe86872360b2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:04.398865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.482540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 316/628] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:04.483768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf76ab-479f-48d7-9a9e-18dbdc8a0f89 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.591101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926e25d-9455-4dd7-9ebb-20343a5ab34e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.630662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.631317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.631601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.631876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.632538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf76ab-479f-48d7-9a9e-18dbdc8a0f89 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.642903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:04.644457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:04.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:04.673552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:04.674540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:04.674540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:04.674540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:04.677637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 313/629] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:04.683426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.700965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.701963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.702459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.702924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.724276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:04.729953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:04.731208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:04.732072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:04.734688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 317/630] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:04.742142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.768513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:04.840339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4586311-2fb8-4b5a-82e7-ca9d46524f10 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.856569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:04.877054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.879058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.879334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.879603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.880317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:04.880625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:04.882059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:04.884440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437e88e-3dc9-4052-b6c7-efedcc19ab49 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:04.897570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 314/631] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:04.898448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:04.935655 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:04.936646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:04.936998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:04.937719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:04.938723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:04.939093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:04.940863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:04.952320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 318/632] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:04.953561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.028192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2381a54-16f0-4914-b0f7-519d03d3cc84 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.036121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.042446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.071695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.072364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.072644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.073077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.078593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d0a1e-bf2a-4fab-85f5-81bca246d545 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.084090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:05.088863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:05.089789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:05.090647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 315/633] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:05.091351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.120000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddd8e1-a736-4b8d-bab8-5358e171eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.120666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddd8e1-a736-4b8d-bab8-5358e171eca9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.121011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddd8e1-a736-4b8d-bab8-5358e171eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.121283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddd8e1-a736-4b8d-bab8-5358e171eca9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.128949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddd8e1-a736-4b8d-bab8-5358e171eca9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.130383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddd8e1-a736-4b8d-bab8-5358e171eca9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-68ddd8e1-a736-4b8d-bab8-5358e171eca9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:05.141635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:05.142647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:05.143608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 319/634] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:49:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:05.144266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d9cff-c87e-4468-aa2e-0121423a6521 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.144266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d9cff-c87e-4468-aa2e-0121423a6521 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.144820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d9cff-c87e-4468-aa2e-0121423a6521 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.148268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 320/635] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:05.148268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d9cff-c87e-4468-aa2e-0121423a6521 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.205775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380b84d-8d36-491d-babc-5c09c7f77a55 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.241982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.242607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.242883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.243150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.253711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.253838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d9cff-c87e-4468-aa2e-0121423a6521 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.255173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:05.270125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:05.271121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:05.271984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:05.271984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:05.271984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:05.271984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 316/636] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:05.272753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.289565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.296491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.302619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab67d4df-318d-4c9a-b59a-198d10247230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.303210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67d4df-318d-4c9a-b59a-198d10247230 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.303481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67d4df-318d-4c9a-b59a-198d10247230 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.303747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab67d4df-318d-4c9a-b59a-198d10247230 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.304426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67d4df-318d-4c9a-b59a-198d10247230 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:05.304714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67d4df-318d-4c9a-b59a-198d10247230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.306072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab67d4df-318d-4c9a-b59a-198d10247230 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.318760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 321/637] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:05.319895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef07b-69c9-4570-8ef3-a6311979f219 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.320201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef07b-69c9-4570-8ef3-a6311979f219 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.320472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ef07b-69c9-4570-8ef3-a6311979f219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.340510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-067ef07b-69c9-4570-8ef3-a6311979f219 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:05.341854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-067ef07b-69c9-4570-8ef3-a6311979f219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:05.383942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815eb25-cf06-4832-b8ab-843601b7b432 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.418145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.418758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.419033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.419301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.420018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:05.420351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.428436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 322/638] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:05.428941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.429033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ef07b-69c9-4570-8ef3-a6311979f219 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.445798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 317/639] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:05.446991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.548640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.557728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ef07b-69c9-4570-8ef3-a6311979f219 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.576503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.579004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec0aaa-cfa4-49ae-ab3f-2ff89334cb92 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.598313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.599450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.599929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.600650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.601463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:05.601932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.603863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.618094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 323/640] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:05.619153 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.619389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.619963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.620333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.620677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.640973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.642786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:05.659772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:05.660755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:05.661649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:05.661649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:05.661649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:05.663121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 318/641] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:05.664339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.727250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860a7f3-b983-498d-83b3-b74274637318 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.763529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.764184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.764441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.764704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.767645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d1e69-61af-4b77-961e-14a82a15d586 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.796477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.803629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.804304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.804580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.804878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.811936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:05.812521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.814137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:05.825538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:05.826561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 05 16:49:05.827497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:05.831543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 319/642] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:05.832423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.840189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.841677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.940126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12dfaf-26a3-490c-8340-d124ab0d61b7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:05.951389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 324/643] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:49:05.951389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:05.983327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:05.983937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:05.984248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:05.984524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:05.985160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:05.985460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:05.986830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:05.999770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 320/644] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:06.004882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.027632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.053542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a16bbae-3956-4553-ad91-0bcb1c898fd6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.056127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.090238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.090877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.091151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.091471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.105750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:06.108957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edd500-d266-4110-b89f-c715e55e7b42 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:06.110480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:06.111385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 325/645] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:06.113765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.149884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.150510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.150794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.151063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.151790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:06.152095 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.153614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.171634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 321/646] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:06.172472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.224140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5d4e3-b8b6-4df2-89b2-ad3289613e8e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.259702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.260360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.260633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.260898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.263561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.271862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.274425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.284198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5266ea7-a38f-450c-b2da-43568e2bb288 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:06.291101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:06.292014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:06.293204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:06.293204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:06.293204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:06.293944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.294604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 326/647] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:06.295489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.323074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.323803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.324140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.324497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.332567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.334091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:06.345706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:06.346850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:49:06.348067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:06.352391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 322/648] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:06.353364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.405912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b09557-1e02-4af0-83d8-467c68ff2503 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.437847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.438478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.438704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.438971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.439621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:06.439920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.441539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.449336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 327/649] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:06.450108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.463250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52910c-36a6-4f4b-a6ad-86f18332723f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.498081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fd33b-ed35-4806-9c90-201b3d41aca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.498624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fd33b-ed35-4806-9c90-201b3d41aca7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.498903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fd33b-ed35-4806-9c90-201b3d41aca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.499170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fd33b-ed35-4806-9c90-201b3d41aca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.499947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.537093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.555116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dde62-e195-4a2e-a91c-a3f58f7e2f4b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.569714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fd33b-ed35-4806-9c90-201b3d41aca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.571176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fd33b-ed35-4806-9c90-201b3d41aca7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.587838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.588483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.588764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.589033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.589706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:06.590001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.591351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.603227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 328/650] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:06.604115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.671077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 323/651] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:49:06.672593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879d29-c697-4fa2-88b9-5354e5e475cd None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.672986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879d29-c697-4fa2-88b9-5354e5e475cd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.673350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879d29-c697-4fa2-88b9-5354e5e475cd None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.674695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 324/652] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:06.676053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879d29-c697-4fa2-88b9-5354e5e475cd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.703309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281e41f-21e4-4ab5-9534-731842c1529f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.713971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.739447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.740051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.740365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.740636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.741368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:06.741668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.741903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.743117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.760564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 329/653] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:06.761382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.781738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879d29-c697-4fa2-88b9-5354e5e475cd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.815566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.816296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.816576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.816847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.826907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.828402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:06.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:06.843801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:06.844841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:06.844841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:06.844841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:06.844841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 325/654] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:06.844841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:06.861416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1840be-722e-4f00-a4f2-84604d8c4060 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.896410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587be91b-d2ec-4b4a-aebd-6dad246c2f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.897032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be91b-d2ec-4b4a-aebd-6dad246c2f87 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.897303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be91b-d2ec-4b4a-aebd-6dad246c2f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.897570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587be91b-d2ec-4b4a-aebd-6dad246c2f87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.904253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587be91b-d2ec-4b4a-aebd-6dad246c2f87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.905714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587be91b-d2ec-4b4a-aebd-6dad246c2f87 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-587be91b-d2ec-4b4a-aebd-6dad246c2f87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:06.916963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:06.918034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:06.919258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 330/655] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:06.919761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59544b28-5f84-4548-939b-9c9d91dc84db None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.919761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59544b28-5f84-4548-939b-9c9d91dc84db None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.920022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59544b28-5f84-4548-939b-9c9d91dc84db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.939146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-59544b28-5f84-4548-939b-9c9d91dc84db None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:06.940369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-59544b28-5f84-4548-939b-9c9d91dc84db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:06.947527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a35ba-a12e-49b5-baca-2258c27efbc9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:06.963778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.967314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:06.984124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:06.984790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:06.985072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:06.985323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:06.996270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:06.997737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:07.012320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:07.013218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:07.014088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:07.014088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:07.014088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:07.014088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 326/656] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:07.014316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 331/657] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:07.014403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.014479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59544b28-5f84-4548-939b-9c9d91dc84db None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.119125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59544b28-5f84-4548-939b-9c9d91dc84db None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.121336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e38a53-7b21-4e01-95be-401684a01176 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.154876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.155483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.155757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.155928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.156047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.156930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.157396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.157855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.158778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:07.159380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.161076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.177540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 332/658] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:07.177540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.197430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.203175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.231795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.233271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.284262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055394a-2f58-4324-9717-ff8830d1ca8d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.316598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.317193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.317472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.317738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.318414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:07.318712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.320126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.332015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 333/659] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:07.332520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.333047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 327/660] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:49:07.333675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.402867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.417170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.433067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d38469-f94e-4861-9172-cfbfc9633c7b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.439133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e37415-1ea0-4e8a-8a4c-d4129ab45a16 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.467456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.468118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.468429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.468706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.472322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.472922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.473197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.473465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.474093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:07.474393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.475739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.475938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.477434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.487015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 334/661] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:07.487897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:07.489478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:07.490394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 328/662] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:07.491347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.587583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300cee4-e021-4342-88ad-f19f55429b4f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.592232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c0de4-7e3e-4be3-8ee6-0e8ca071d22c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.621426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.622050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.622327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.622597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.623126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.628653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.629264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.629576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.629851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.630080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.630575 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=c30e20ce4d63452095aedb60249f0cce) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:07.630878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.632345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.635133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:07.638452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 329/663] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/role_assignments?user.id=c30e20ce4d63452095aedb60249f0cce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:07.639279 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:07.640200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:07.641307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:07.642331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:07.644717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 335/664] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:07.645745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.754987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec99361-ff85-42a4-9c01-92078390d019 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.771639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caba1a8-f08e-4aa7-83b5-a8684434372c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.796235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.796235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.796235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.796235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.808940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.810405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.811888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.812623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.812879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.813146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.821493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.822919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:07.824829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:07.825728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:07.826608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:07.826608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:07.826608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:07.826608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 330/665] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:07.827005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:07.836444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:07.837355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:07.838281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:07.838718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 336/666] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:07.839359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:07.892142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.905322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:07.943087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0195f-0153-4354-a3ea-6621ae1947fc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.949679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee5dd1-738e-46ff-abc0-0f7160b33499 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:07.978020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.978639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.978916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.979190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.979913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:07.980253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.982011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:07.988052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:07.988954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:07.989299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:07.989572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:07.990264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:07.990573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:07.992262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.000397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 331/667] 10.222.0.23 () {56 vars in 1139 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:08.005653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 337/668] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:08.005653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.006276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.109489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b338-1db0-45ff-a5c7-909ee77221fb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.113389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c16828-b9fe-41be-80b6-f4a55bd4e722 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.117999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.134465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.147668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.148341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.148643 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.148919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.149564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:08.149862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.151269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.151538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.152163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.152438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.152812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.153380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:08.154032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.155085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.163999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 332/669] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:08.172414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 338/670] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:08.172556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.172647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.276500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e692f4-5b4b-4e21-8357-c0989f918042 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.282893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f4ddf-46b0-4702-bea0-a32dec501d27 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.314395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.315294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.315828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.316104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.317911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.318539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.318811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.319075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.329315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.330757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:08.346238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:08.347863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:08.348942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:08.348942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:08.348942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:08.349272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 339/671] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:08.350089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.360228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.393401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.406827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.408308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.466694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764463-2408-42d2-9efb-d8d600790149 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.502463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.503153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.503419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.503688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.510679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.512345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.519724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 333/672] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:49:08.519724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:08.524018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:08.524935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:08.525839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 340/673] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:08.526341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.595122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.624332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.634336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65aeb9-87c3-47b1-90b4-4d184349dafe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.634750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed4b00-69b2-46a8-9158-193e62c92948 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.670199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.670812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.671094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.671360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.672003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:08.672347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.672602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.673415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.673710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.674090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.674546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.676035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=05a016553950460dbb736ced2cae4f62) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:08.676435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.677909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.690128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 334/674] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/role_assignments?user.id=05a016553950460dbb736ced2cae4f62&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:08.690128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.690444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 341/675] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:08.690444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.816336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddc545-0953-4cbc-a8c6-414ff6c3dd62 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.844387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af3e21-43ec-4a2d-b745-aa705ca29a6d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:08.860442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.861313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.861595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.861920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.862924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:08.863236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.864740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.883665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 342/676] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:08.883665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:08.888755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.897828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:08.898906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:08.899027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:08.899776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:08.900372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:08.915131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:08.916916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:08.940995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:08.941965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:08.942841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:08.942841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:08.942841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:08.942841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 335/677] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:08.944341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.020705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722703ce-ec25-4ac7-8369-eac7bd530635 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.065084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfd0a0-8b2d-4146-8e54-7390c5e1240f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.066051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.067218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.068466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.068578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.082619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.085043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:09.100874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:09.101841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:09.102766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:09.103342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.103342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.103342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.103342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.104166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=36c795d7c6bc4c519c15f32489c44583) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:09.104166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.105854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.106509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 343/678] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:09.107591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.113441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 336/679] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:08 2022] GET /identity/v3/role_assignments?user.id=36c795d7c6bc4c519c15f32489c44583&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:09.113705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.165813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:09.197240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:09.225584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7697bc-8086-4020-8ba9-742b09c942f3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.233075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148bd72-3a9a-467e-904a-42a6aef8f928 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.262954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.263607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.263883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.264242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.264978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:09.265286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.266820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.274374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.275389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.275893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.276399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.277352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:09.277864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.279524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.288636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 337/680] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Sep 5 16:49:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:09.293881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 344/681] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:09.298927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.299084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.421502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747843-e339-46ff-ada1-b99578008140 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.426059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:09.468521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.469572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.470120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.470560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.471933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782a33-9365-4099-9de7-5796b90ce0a0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.479816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.482589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:09.497098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:09.498037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:09.498937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:09.499547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 338/682] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:09.505027 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.511730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:09.522082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.523267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.524011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.524758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.618079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.619686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.630944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d7e94-da9f-4325-998f-b1d61a4c0fd0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.672379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ca346-8480-4c6c-b937-66d1fe416a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.673518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca346-8480-4c6c-b937-66d1fe416a8c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.673518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca346-8480-4c6c-b937-66d1fe416a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.673518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca346-8480-4c6c-b937-66d1fe416a8c None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.674081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ca346-8480-4c6c-b937-66d1fe416a8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:09.674385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ca346-8480-4c6c-b937-66d1fe416a8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.675975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ca346-8480-4c6c-b937-66d1fe416a8c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.689984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 339/683] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:09.691311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15ac1-fa28-4997-b7c1-bac4ded7369b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.691686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15ac1-fa28-4997-b7c1-bac4ded7369b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.691963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e15ac1-fa28-4997-b7c1-bac4ded7369b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.693403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 340/684] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:09.699434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45cb67-66b6-4e1a-9ffc-77fd999a379b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.699989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45cb67-66b6-4e1a-9ffc-77fd999a379b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.700380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45cb67-66b6-4e1a-9ffc-77fd999a379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.722507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-be45cb67-66b6-4e1a-9ffc-77fd999a379b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:09.723916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-be45cb67-66b6-4e1a-9ffc-77fd999a379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:09.746421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:09.759174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 345/685] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:49:09.760646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.800078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:09.814193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 341/686] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:09.815250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45cb67-66b6-4e1a-9ffc-77fd999a379b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:09.926140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f3759-9057-4619-82ab-0c8a187100d6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:09.990023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:09.990634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:09.990907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:09.991175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:09.991901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=fcbcd7638cc84cdd87a1eb98f5926425) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:09.992224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:09.993643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:09.994103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45cb67-66b6-4e1a-9ffc-77fd999a379b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.000427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 346/687] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:09 2022] GET /identity/v3/role_assignments?user.id=fcbcd7638cc84cdd87a1eb98f5926425&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:10.063517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.064215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.064501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.064775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.084966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:10.086475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:10.106819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:10.107835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:10.109171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:10.109171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:10.109171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:10.109171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 342/688] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:10.113795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.149005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.162569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:10.238265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:10.244522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144d37a-3da2-444b-b650-4abc5fc8493e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.275894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e686b10-3eb9-4807-8270-f602f364463f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.281316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.281931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.282206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.282471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.283100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:10.283397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:10.284973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:10.298167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 347/689] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:10.303364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.318141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.318779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.319055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.319324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.331414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:10.337170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:10.338081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:10.338903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:10.344089 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 343/690] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:10.344908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.423145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:10.441644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11136a0e-4135-4a70-92c4-7768ed2145ec None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.465813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791539b-f347-4442-8f9c-3ba3aa0d994a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.492420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.493121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.493493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.493859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.504375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:10.506293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:10.509125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.509988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.510473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.510911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.511737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:10.512304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:10.513936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:10.519481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:10.520420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:10.521471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 348/691] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:10.526292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.532927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 344/692] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:10.539950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.593868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:10.644910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c24a5e-673f-4192-8332-56ccc02cfb48 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.685663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:10.701878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.703088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.703365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.703629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.725436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:10.726898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:10.754212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:10.755518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:10.756508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:10.756508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:10.756508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:10.756508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 345/693] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:10.762920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.782464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e519d-9db4-4ed0-88bd-1902d2531c6b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.849310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.849929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.850205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.850465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.851410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:10.851722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:10.853430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:10.878918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 349/694] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:10.885194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:10.929894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bcc69-80a5-4c75-9f29-0683d9c0af44 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:10.977331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6d73c-970d-4aec-87f3-9ce1d46c3eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:10.979419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6d73c-970d-4aec-87f3-9ce1d46c3eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:10.979908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6d73c-970d-4aec-87f3-9ce1d46c3eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:10.980741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6d73c-970d-4aec-87f3-9ce1d46c3eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:10.982435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6d73c-970d-4aec-87f3-9ce1d46c3eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:10.983489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6d73c-970d-4aec-87f3-9ce1d46c3eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:10.987714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6d73c-970d-4aec-87f3-9ce1d46c3eb1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:11.010165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 346/695] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:11.011374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5444-33fb-42f7-a1a7-ce95ea6ca6c0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.015822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5444-33fb-42f7-a1a7-ce95ea6ca6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.015822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5444-33fb-42f7-a1a7-ce95ea6ca6c0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.015822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 347/696] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:11.017952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5444-33fb-42f7-a1a7-ce95ea6ca6c0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:11.032814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:11.058530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222744-e3bc-44d8-8cb7-fd0c66ab3ff8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:11.074680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:11.100246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f5f17-025d-4799-b537-a0e1415ddd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:11.100998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f5f17-025d-4799-b537-a0e1415ddd0e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.101363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f5f17-025d-4799-b537-a0e1415ddd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f5f17-025d-4799-b537-a0e1415ddd0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.102573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f5f17-025d-4799-b537-a0e1415ddd0e None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2756036297064cc19d5fd5d1861703c3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:11.102970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f5f17-025d-4799-b537-a0e1415ddd0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:11.104559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f5f17-025d-4799-b537-a0e1415ddd0e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:11.111383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 350/697] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:10 2022] GET /identity/v3/role_assignments?user.id=2756036297064cc19d5fd5d1861703c3&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:11.112750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ab64-af5c-4c1a-907a-1ef575a8f1d2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.113302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ab64-af5c-4c1a-907a-1ef575a8f1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.113739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ab64-af5c-4c1a-907a-1ef575a8f1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.148438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c421ab64-af5c-4c1a-907a-1ef575a8f1d2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:11.149849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c421ab64-af5c-4c1a-907a-1ef575a8f1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:11.164142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5444-33fb-42f7-a1a7-ce95ea6ca6c0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:11.241856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:11.242872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.243401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.244576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.262800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:11.264595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:11.285183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:11.286214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:11.287172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:11.293600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 348/698] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:49:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:11.294990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:11.298256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 351/699] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:11.411865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:11.415328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d5a10-a677-4493-b3cc-f03319f98da0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:11.453339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:11.453965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.454244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.454513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.455249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:11.455563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:11.457069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:11.476135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 349/700] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:49:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:11.482053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421ab64-af5c-4c1a-907a-1ef575a8f1d2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:11.569214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:11.592854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:11.610130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421ab64-af5c-4c1a-907a-1ef575a8f1d2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:11.681713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf46f22-ac96-4806-891f-356f17f3bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:11.682621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf46f22-ac96-4806-891f-356f17f3bf79 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.683059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf46f22-ac96-4806-891f-356f17f3bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.683475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf46f22-ac96-4806-891f-356f17f3bf79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.742424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098c223-a4c4-4123-89a2-5dce7fcbbbb3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:11.760328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf46f22-ac96-4806-891f-356f17f3bf79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:11.761778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf46f22-ac96-4806-891f-356f17f3bf79 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:11.778984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a1b47-ec90-4a9f-a03c-bfa826ed47a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:11.779850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1b47-ec90-4a9f-a03c-bfa826ed47a6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.780324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1b47-ec90-4a9f-a03c-bfa826ed47a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.780746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1b47-ec90-4a9f-a03c-bfa826ed47a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.791616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a1b47-ec90-4a9f-a03c-bfa826ed47a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:11.793285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a1b47-ec90-4a9f-a03c-bfa826ed47a6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-109a1b47-ec90-4a9f-a03c-bfa826ed47a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:11.808663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:11.809879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:11.811000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:11.811000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:11.811000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:11.812805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 350/701] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:11.818777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2abf0-20d5-409a-9a26-501b76fc7577 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.819357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2abf0-20d5-409a-9a26-501b76fc7577 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.819792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2abf0-20d5-409a-9a26-501b76fc7577 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.821542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 351/702] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:11.822428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2abf0-20d5-409a-9a26-501b76fc7577 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:11.835692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:11.837856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:11.877079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 352/703] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 05 16:49:11.878641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d06039-34ff-43ae-b5e3-3add02b7ff4a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.878910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d06039-34ff-43ae-b5e3-3add02b7ff4a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.879274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d06039-34ff-43ae-b5e3-3add02b7ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.898102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f9d06039-34ff-43ae-b5e3-3add02b7ff4a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:11.899345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f9d06039-34ff-43ae-b5e3-3add02b7ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:11.937944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2abf0-20d5-409a-9a26-501b76fc7577 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:11.973821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:11.974784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:11.975208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:11.975622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:11.976531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:11.977079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:11.978687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:11.984589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 353/704] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:11.985557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d06039-34ff-43ae-b5e3-3add02b7ff4a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:11.998323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 352/705] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:12.012177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:12.085998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:12.125333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381b411-b0ae-4b02-9fbc-8e9a8ed941de None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:12.164401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:12.165412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.165884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.166367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.167299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:12.168998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:12.199148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 353/706] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:49:12 2022] POST /identity/v3/users => generated 254 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:49:12.209708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:12.213084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d06039-34ff-43ae-b5e3-3add02b7ff4a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:12.273447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:12.274591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.275062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.275521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.290831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:12.292943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:12.315917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:12.317041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:12.317942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:12.317942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:12.317942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:12.319936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 354/707] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:12.325620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:12.372554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:12.474781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0203193b-aa86-41f7-9c79-73b8e7fb7561 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:12.496592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d4ca6-6331-4199-aefa-a14e9edd27d6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:12.512763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:12.526782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a44fb-6f1e-4fd8-a325-f092a9393e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:12.528093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a44fb-6f1e-4fd8-a325-f092a9393e92 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.528595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a44fb-6f1e-4fd8-a325-f092a9393e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.529299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a44fb-6f1e-4fd8-a325-f092a9393e92 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.530170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a44fb-6f1e-4fd8-a325-f092a9393e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:12.530893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a44fb-6f1e-4fd8-a325-f092a9393e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:12.533620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a44fb-6f1e-4fd8-a325-f092a9393e92 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:12.540529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404523a6-d5a1-4586-9059-eda5e63b24d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:12.541246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404523a6-d5a1-4586-9059-eda5e63b24d1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.541527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404523a6-d5a1-4586-9059-eda5e63b24d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.541795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404523a6-d5a1-4586-9059-eda5e63b24d1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.542397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404523a6-d5a1-4586-9059-eda5e63b24d1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:12.543764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404523a6-d5a1-4586-9059-eda5e63b24d1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:12.549183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 354/708] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:12.550527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf52e9c-d1e3-4763-9526-d4f82e0c0200 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.550810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf52e9c-d1e3-4763-9526-d4f82e0c0200 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.551078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf52e9c-d1e3-4763-9526-d4f82e0c0200 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.552294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 355/709] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:12.556962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf52e9c-d1e3-4763-9526-d4f82e0c0200 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-404523a6-d5a1-4586-9059-eda5e63b24d1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:12.559208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:12.560230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:12.561055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:12.562284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 355/710] 10.222.0.23 () {56 vars in 1129 bytes} [Mon Sep 5 16:49:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:12.563173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd38cb-fef0-437f-b0dc-2b8d3cc6cfd1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.563455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd38cb-fef0-437f-b0dc-2b8d3cc6cfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.563726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd38cb-fef0-437f-b0dc-2b8d3cc6cfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.583868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-75dd38cb-fef0-437f-b0dc-2b8d3cc6cfd1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:12.585112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-75dd38cb-fef0-437f-b0dc-2b8d3cc6cfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:12.597679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:12.665064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 356/711] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:12.665854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd38cb-fef0-437f-b0dc-2b8d3cc6cfd1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:12.671900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf52e9c-d1e3-4763-9526-d4f82e0c0200 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:12.707629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:12.708312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.708734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.709005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.716821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:12.718371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:12.732046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:12.733092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:12.733951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:12.738722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 356/712] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:12.739938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:12.776070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd38cb-fef0-437f-b0dc-2b8d3cc6cfd1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:12.787501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:12.815453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:12.816470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.817028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.817451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.818333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:12.818799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:12.820450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:12.825946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 357/713] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Sep 5 16:49:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:12.836296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:12.865105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:12.865468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105b4b9-ef2e-4823-91d1-f511df0bd8bc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:12.918048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:12.918792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:12.919068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:12.920937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:12.921637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:12.921946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:12.923976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:12.949539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 357/714] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:12.957135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:13.055445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e7145-1439-4037-8ddc-75862e6fa865 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:13.075872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4a391-aded-4305-9683-07cd7a3a05a0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:13.084683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:13.097529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:13.098145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:13.098418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:13.098687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:13.099315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:13.099616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:13.101000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:13.109480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 358/715] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Sep 5 16:49:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:13.110395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:13.116842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531fd94-3533-4111-9869-93862128de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:13.117739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531fd94-3533-4111-9869-93862128de53 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:13.118167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531fd94-3533-4111-9869-93862128de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:13.118625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531fd94-3533-4111-9869-93862128de53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:13.119520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531fd94-3533-4111-9869-93862128de53 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=c30e20ce4d63452095aedb60249f0cce) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:13.119984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531fd94-3533-4111-9869-93862128de53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:13.121807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531fd94-3533-4111-9869-93862128de53 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:13.128689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 358/716] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:12 2022] GET /identity/v3/role_assignments?user.id=c30e20ce4d63452095aedb60249f0cce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:13.199428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:13.232677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53349743-1cfc-4747-946c-aa5515d53024 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:13.272692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:13.273345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:13.273624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:13.273892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] PATH_INFO: `/v3/services/network` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:13.274475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:13.275864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:13.286238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:13.287512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:13.288607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 359/717] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:13 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:13.295588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531fd94-3533-4111-9869-93862128de53 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:13.426284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531fd94-3533-4111-9869-93862128de53 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:13.465096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6b48-dbfc-42f4-acd7-1b77de6ebe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:13.466068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6b48-dbfc-42f4-acd7-1b77de6ebe52 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:13.466548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6b48-dbfc-42f4-acd7-1b77de6ebe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:13.467103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6b48-dbfc-42f4-acd7-1b77de6ebe52 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:13.467955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6b48-dbfc-42f4-acd7-1b77de6ebe52 None admin] RBAC: Adding query filter params (name=network) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:13.468481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6b48-dbfc-42f4-acd7-1b77de6ebe52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:13.470061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6b48-dbfc-42f4-acd7-1b77de6ebe52 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:13.475581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 359/718] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:13.482195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:13.615499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265a90c-3ec8-430f-b888-b58f75e3ee84 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:13.655178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0055754-fa8d-42cc-90b7-676060de4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:13.655797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0055754-fa8d-42cc-90b7-676060de4a7e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:13.656075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0055754-fa8d-42cc-90b7-676060de4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:13.656366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0055754-fa8d-42cc-90b7-676060de4a7e None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:13.656998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0055754-fa8d-42cc-90b7-676060de4a7e None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:13.657297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0055754-fa8d-42cc-90b7-676060de4a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:13.658669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0055754-fa8d-42cc-90b7-676060de4a7e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:13.668674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 360/719] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:13.756936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf9b9d-711f-4078-81f1-e1b4257fe996 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:13.757133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf9b9d-711f-4078-81f1-e1b4257fe996 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:13.757398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf9b9d-711f-4078-81f1-e1b4257fe996 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:13.759009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 360/720] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:13.764895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2314f38-e52e-4d74-b7ac-721e6719a135 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:13.765250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2314f38-e52e-4d74-b7ac-721e6719a135 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:13.765434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2314f38-e52e-4d74-b7ac-721e6719a135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:13.775254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:13.793378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a2314f38-e52e-4d74-b7ac-721e6719a135 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:13.794617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a2314f38-e52e-4d74-b7ac-721e6719a135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:13.883651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 361/721] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:14.217051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf9b9d-711f-4078-81f1-e1b4257fe996 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:14.271264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:14.351292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf9b9d-711f-4078-81f1-e1b4257fe996 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:14.402073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:14.403061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:14.403540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:14.404064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:14.417488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:14.422831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:14.423808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:14.424725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:14.427551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 361/722] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:14.433006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2314f38-e52e-4d74-b7ac-721e6719a135 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:14.570761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2314f38-e52e-4d74-b7ac-721e6719a135 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:14.607848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:14.616028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb504f0c-65eb-4186-b141-1ad6f9562fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:14.616678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504f0c-65eb-4186-b141-1ad6f9562fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:14.616963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504f0c-65eb-4186-b141-1ad6f9562fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:14.617240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb504f0c-65eb-4186-b141-1ad6f9562fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:14.617913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb504f0c-65eb-4186-b141-1ad6f9562fb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:14.618258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb504f0c-65eb-4186-b141-1ad6f9562fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:14.619745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb504f0c-65eb-4186-b141-1ad6f9562fb4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:14.629735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 362/723] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:14.634895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:14.642730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c50d2-f488-4021-a977-c7af40866506 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:14.643008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c50d2-f488-4021-a977-c7af40866506 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:14.648186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c50d2-f488-4021-a977-c7af40866506 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:14.648186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 363/724] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:14.666188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a48a-3522-4347-a141-6c52f11d3586 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:14.666566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a48a-3522-4347-a141-6c52f11d3586 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:14.666743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580a48a-3522-4347-a141-6c52f11d3586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:14.697986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2580a48a-3522-4347-a141-6c52f11d3586 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:14.699241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2580a48a-3522-4347-a141-6c52f11d3586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:14.765675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:14.774885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e58aa2-9711-43c4-8b8c-0e65c6e76783 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:14.814060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:14.814696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:14.814971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:14.815239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:14.831631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 364/725] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:14.836097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:14.837571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:14.852486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:14.853434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:14.854319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:14.854319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:14.854319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:14.854319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 362/726] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:14.859488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580a48a-3522-4347-a141-6c52f11d3586 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:15.006923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580a48a-3522-4347-a141-6c52f11d3586 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:15.039949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:15.052572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:15.053204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.053478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.053746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.054408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:15.054706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:15.056090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:15.077475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 365/727] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:15.082947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:15.222270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:15.227793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb4ed6-f395-4231-9071-80c16e21af75 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:15.300297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:15.300959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.301292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.301565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.314609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:15.316118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:15.336557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:15.337508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:15.338434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:15.343630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 363/728] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:15.345098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:15.358987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:15.394105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa6cc02-74d5-4631-99e6-72f86e576184 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:15.431630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:15.432721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.433276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.433750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.445481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:15.451162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:15.452078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:15.452969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:15.456015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 366/729] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:15.462078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:15.543875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:15.561812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652379e-d877-4761-b323-79cbe355d54d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:15.637313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:15.637926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.638172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.639026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.639962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:15.640300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:15.641748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:15.656318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059b9c9-5791-4508-a8d7-1570dc8cd7e8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:15.684285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 364/730] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Sep 5 16:49:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:15.687091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:15.719808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa2221-33a5-445b-bf3b-1b49149cf744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:15.720455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa2221-33a5-445b-bf3b-1b49149cf744 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.720726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa2221-33a5-445b-bf3b-1b49149cf744 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.721008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa2221-33a5-445b-bf3b-1b49149cf744 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.721803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa2221-33a5-445b-bf3b-1b49149cf744 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:15.722171 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa2221-33a5-445b-bf3b-1b49149cf744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:15.723549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa2221-33a5-445b-bf3b-1b49149cf744 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:15.733080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 367/731] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:15.734080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c9891-ef47-4af2-b760-5ab6b939a12f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.734351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c9891-ef47-4af2-b760-5ab6b939a12f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.734633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c9891-ef47-4af2-b760-5ab6b939a12f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.735944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 368/732] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:15.741203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8306cf-f304-410f-9dc9-1c28b5416427 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.741479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8306cf-f304-410f-9dc9-1c28b5416427 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.741746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8306cf-f304-410f-9dc9-1c28b5416427 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.744209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 369/733] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:15.744896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd14d-071a-45c5-a523-dd38ff248627 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.745175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd14d-071a-45c5-a523-dd38ff248627 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.745444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd14d-071a-45c5-a523-dd38ff248627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.765327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ec4fd14d-071a-45c5-a523-dd38ff248627 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:15.766241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:15.766508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ec4fd14d-071a-45c5-a523-dd38ff248627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:15.805735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc9e4b-26aa-4a11-9d28-35981e4a1915 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:15.835255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 370/734] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:15.836091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fd14d-071a-45c5-a523-dd38ff248627 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:15.850460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff4a33-f1ab-4107-ba12-16e610f5bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:15.851479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff4a33-f1ab-4107-ba12-16e610f5bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:15.851758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff4a33-f1ab-4107-ba12-16e610f5bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:15.852029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff4a33-f1ab-4107-ba12-16e610f5bbf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:15.879413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:15.943513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff4a33-f1ab-4107-ba12-16e610f5bbf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:15.944957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff4a33-f1ab-4107-ba12-16e610f5bbf6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.014587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:16.058567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 365/735] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 05 16:49:16.060485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec55e53-d546-44a1-8fea-7bbb0c81960e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.061093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec55e53-d546-44a1-8fea-7bbb0c81960e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.061897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec55e53-d546-44a1-8fea-7bbb0c81960e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.070576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fd14d-071a-45c5-a523-dd38ff248627 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.080504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1ec55e53-d546-44a1-8fea-7bbb0c81960e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:16.081759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1ec55e53-d546-44a1-8fea-7bbb0c81960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:16.106473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0da03-b5a1-4373-9e02-7d882ee2a61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.107112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0da03-b5a1-4373-9e02-7d882ee2a61c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.107387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0da03-b5a1-4373-9e02-7d882ee2a61c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.107659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0da03-b5a1-4373-9e02-7d882ee2a61c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.117737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0da03-b5a1-4373-9e02-7d882ee2a61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.119180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0da03-b5a1-4373-9e02-7d882ee2a61c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d6c0da03-b5a1-4373-9e02-7d882ee2a61c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:16.133601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:16.134529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:16.135400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:16.135400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:16.135400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:16.135400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 371/736] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:16.136077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c73d47-12bd-40ab-8bd9-6db2edda8ef8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.136220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c73d47-12bd-40ab-8bd9-6db2edda8ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.136421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c73d47-12bd-40ab-8bd9-6db2edda8ef8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.137656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 372/737] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:16.138343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c73d47-12bd-40ab-8bd9-6db2edda8ef8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.153557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 366/738] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:16.154668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec55e53-d546-44a1-8fea-7bbb0c81960e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.251061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:16.260908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:16.264713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c73d47-12bd-40ab-8bd9-6db2edda8ef8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.268875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec55e53-d546-44a1-8fea-7bbb0c81960e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.299091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d0dd0-7f7b-42ac-9d76-9b0cb106652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.299719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d0dd0-7f7b-42ac-9d76-9b0cb106652b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.300052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d0dd0-7f7b-42ac-9d76-9b0cb106652b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.300527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d0dd0-7f7b-42ac-9d76-9b0cb106652b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.308001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.308667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.308945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.309210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.311104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d0dd0-7f7b-42ac-9d76-9b0cb106652b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.312590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d0dd0-7f7b-42ac-9d76-9b0cb106652b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.319349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:16.324141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:16.325103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:16.325940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 367/739] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:16.326647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-097d0dd0-7f7b-42ac-9d76-9b0cb106652b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:16.332298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:16.333262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:16.334129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:16.334129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:16.334129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:16.334129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 373/740] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:16.335620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea608-7b78-433d-89f9-9520841a6c7f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.335901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea608-7b78-433d-89f9-9520841a6c7f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.336181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ea608-7b78-433d-89f9-9520841a6c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.355095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cf4ea608-7b78-433d-89f9-9520841a6c7f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:16.356648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf4ea608-7b78-433d-89f9-9520841a6c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:16.428759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 374/741] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 05 16:49:16.429595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ea608-7b78-433d-89f9-9520841a6c7f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.440701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f9f12-7d87-4f28-80e1-7fe3c59a919d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.476274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.476881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.477163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.477438 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.478199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:16.478507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.480029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.492642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 368/742] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:16.493473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.503758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:16.513205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:16.538257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ea608-7b78-433d-89f9-9520841a6c7f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.574129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.575185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.575466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.575737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.576777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:16.577092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.578765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.592548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 375/743] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:16.593375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.599317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e4468-6af4-4c32-999a-7629265b8950 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.636521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.637250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.637525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.638087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.638416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:16.639019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.640453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.652874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 369/744] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:16.653665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.702812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d30b9-ec16-48ad-98ea-79c7efc94f4e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.727255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:16.731700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:16.736830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.737467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.737740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.738174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.738825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.740202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.755550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2a092-031d-468c-ace6-325cd3fb10e5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.756118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 376/745] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:16 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:16.756959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.813922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.814579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.814846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.815272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.824782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.826256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:16.838992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:16.840106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 370/746] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:16.846509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.882492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885253-3aeb-467f-a393-4b3030753966 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.923648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:16.924324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:16.925413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:16.925413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:16.936676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:16.938172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:16.953632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:16.954827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:16.955762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:16.955762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:16.955762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:16.955762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 377/747] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:16.955991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:16.962213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda64f88-f527-4072-b4fb-738384f21863 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:16.994480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.001732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.002444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.003236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.003579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.008887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.012268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.013809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:17.026147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:17.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:17.028499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 371/748] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:17.028990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.073767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68fcb7-4d7f-408b-b82a-75b3f3b73cae None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.108709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.109699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.109978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.110245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.111243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.113430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.129062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 378/749] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:16 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:17.130306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.142798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310b86f-8f5b-4c42-a904-02904fd67f89 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.191503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.192137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.192490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.192766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.193614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:17.194526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.196610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.224555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 372/750] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:17.225451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.281252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.283232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65df2c-eef7-4b51-b055-012dc2532bc7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.287349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.333002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.333635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.333916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.334188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.334873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:17.335177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.337031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.350404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 379/751] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:17.351250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.363734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b6a59-80b7-49f4-bf5b-9b7383638a1d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.411947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.412740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.413121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.413401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.414170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:17.414476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.416227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.432692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 373/752] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:17.433538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.463637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c49c17-0c88-400b-a73d-a5d8de68870c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.500497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.501383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.501795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.502197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.535284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.570000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab506-7a33-4255-a589-694ae32ca6da None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.572673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.576408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.578058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.613700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.614619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.615033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.615437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.623702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.625395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:17.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:17.638976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:17.640084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:17.642164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 374/753] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:17.643093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.702744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 380/754] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:17.703860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.750876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7ad47-ee97-476e-b2da-2cf2e2c7725b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.785105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.799322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.800659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.801036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.801401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.802225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2756036297064cc19d5fd5d1861703c3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:17.803115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.805108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.816304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 375/755] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/role_assignments?user.id=2756036297064cc19d5fd5d1861703c3&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:17.817471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:17.844517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf059cc1-5c59-4534-99c6-b2646739e06c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:17.882608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:17.919015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:17.919639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:17.921010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:17.921194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:17.921907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:17.922211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:17.923646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:17.960978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 381/756] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:17.967490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.023241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ef3fe-4620-48b7-b917-4079b66f0dd9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.082354 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.083270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.084157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.084653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.108382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.110138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:18.138824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:18.139996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:18.141119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:18.141119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:18.141119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:18.143796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 376/757] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:18.153040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0b5e0-57b8-45b5-a369-7464a300f917 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.153234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.184547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:18.192792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.195021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.195540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.196195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.215137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.217523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:18.245605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:18.246612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:18.247474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:18.247474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:18.247474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:18.258210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 382/758] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:18.259780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.323060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:18.373487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233de3a-39fd-4f03-80d7-fd15db3f065c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.422764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30789002-0e45-4fb3-a75a-4b7f8c34cea2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.447665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.449912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.450576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.451706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.452593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:18.453290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.455599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.479140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 377/759] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:18.485966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.490070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.491156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.492552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.492830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.493650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:18.493961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.495339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.515183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 383/760] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:18.520585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.585214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:18.674112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271138a-c38a-4543-ab5c-da387d537aa8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.694273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34315a3-a432-4e4c-8c15-1721925928e7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.711810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.713113 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.713579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.714055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.729303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.731656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.742107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5519a5-c78b-4538-885b-1e67b2fe535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.742753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5519a5-c78b-4538-885b-1e67b2fe535c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.743033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5519a5-c78b-4538-885b-1e67b2fe535c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.743361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5519a5-c78b-4538-885b-1e67b2fe535c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:18.749250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:18.750096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:18.750967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 384/761] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:18.757255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5519a5-c78b-4538-885b-1e67b2fe535c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.758732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5519a5-c78b-4538-885b-1e67b2fe535c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.760039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3b5519a5-c78b-4538-885b-1e67b2fe535c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:18.772446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:18.773333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 378/762] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:18.775108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efee0cc-ee2f-4b69-b1a1-9d2a040a6e20 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.776394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efee0cc-ee2f-4b69-b1a1-9d2a040a6e20 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.776394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efee0cc-ee2f-4b69-b1a1-9d2a040a6e20 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.781620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 379/763] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:18.782026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:18.783663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efee0cc-ee2f-4b69-b1a1-9d2a040a6e20 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.879013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:18.888543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dd5e6-473e-463f-9ad5-b0c2d27ef818 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.914306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efee0cc-ee2f-4b69-b1a1-9d2a040a6e20 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:18.925809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8efb43-f33b-439e-9870-e0dc1bbed8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.926455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8efb43-f33b-439e-9870-e0dc1bbed8fa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.926804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8efb43-f33b-439e-9870-e0dc1bbed8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.927134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8efb43-f33b-439e-9870-e0dc1bbed8fa None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.927941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8efb43-f33b-439e-9870-e0dc1bbed8fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:18.928327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8efb43-f33b-439e-9870-e0dc1bbed8fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.929871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8efb43-f33b-439e-9870-e0dc1bbed8fa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.948340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 385/764] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:18.949769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aecb08-f3f4-4a95-a276-cbfa90f7d43c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.950122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aecb08-f3f4-4a95-a276-cbfa90f7d43c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.950484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aecb08-f3f4-4a95-a276-cbfa90f7d43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.954529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:18.955150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:18.955432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:18.955706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:18.956573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:18.956890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:18.958331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:18.973165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 380/765] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:18.978954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:18.979111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f3aecb08-f3f4-4a95-a276-cbfa90f7d43c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:18.979111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3aecb08-f3f4-4a95-a276-cbfa90f7d43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:19.039601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:19.054010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 386/766] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:19.055140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aecb08-f3f4-4a95-a276-cbfa90f7d43c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:19.082320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ef006-c723-4126-a35e-1a14e3d7419d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:19.120425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442200b-9923-4189-9549-3cf2c8a49220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:19.121105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442200b-9923-4189-9549-3cf2c8a49220 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:19.121387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442200b-9923-4189-9549-3cf2c8a49220 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:19.121972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442200b-9923-4189-9549-3cf2c8a49220 None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/c30e20ce4d63452095aedb60249f0cce/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:19.124319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442200b-9923-4189-9549-3cf2c8a49220 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=c30e20ce4d63452095aedb60249f0cce, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:19.126359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442200b-9923-4189-9549-3cf2c8a49220 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:19.146922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 381/767] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:18 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/c30e20ce4d63452095aedb60249f0cce/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:19.152551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:19.172858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aecb08-f3f4-4a95-a276-cbfa90f7d43c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:19.214411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4be65-6c54-4904-bd82-93d9086a9d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:19.215362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4be65-6c54-4904-bd82-93d9086a9d44 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:19.215860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4be65-6c54-4904-bd82-93d9086a9d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:19.216299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4be65-6c54-4904-bd82-93d9086a9d44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:19.217208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4be65-6c54-4904-bd82-93d9086a9d44 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2f2310a3da094b959270d797bd2f1553) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:19.217649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4be65-6c54-4904-bd82-93d9086a9d44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:19.219289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4be65-6c54-4904-bd82-93d9086a9d44 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:19.226481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 387/768] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:19 2022] GET /identity/v3/role_assignments?user.id=2f2310a3da094b959270d797bd2f1553&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:19.329674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d201-48af-4dd6-998e-f2a174ebbe52 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:19.330021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d201-48af-4dd6-998e-f2a174ebbe52 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:19.330229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d201-48af-4dd6-998e-f2a174ebbe52 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:19.332047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 382/769] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:19.338065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514a17a-684a-44c3-a57a-5e2f79663337 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:19.338418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514a17a-684a-44c3-a57a-5e2f79663337 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:19.338610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514a17a-684a-44c3-a57a-5e2f79663337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:19.358823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7514a17a-684a-44c3-a57a-5e2f79663337 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:19.360024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7514a17a-684a-44c3-a57a-5e2f79663337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:19.444960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 388/770] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:19.450353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959d201-48af-4dd6-998e-f2a174ebbe52 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:19.559070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959d201-48af-4dd6-998e-f2a174ebbe52 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:19.580077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:19.599246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:19.600461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:19.601070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:19.601777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:19.602379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:19.603782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:19.615441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:19.616509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:19.617356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:19.620498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 383/771] 10.222.0.23 () {56 vars in 1129 bytes} [Mon Sep 5 16:49:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:19.622790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514a17a-684a-44c3-a57a-5e2f79663337 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:19.737046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:19.770775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514a17a-684a-44c3-a57a-5e2f79663337 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:19.818151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:19.820398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:19.821021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:19.821407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:19.821682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:19.822407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:19.822711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:19.824091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:19.832177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 389/772] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Sep 5 16:49:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:19.836625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:19.963351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2f5b8-e938-4b31-85e6-83117d67a096 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:19.980964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda6a8-d254-4ae6-8299-8df26bd3d5d5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.003436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e039084-70f2-43d9-b5bf-b5e30b8101a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.004447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039084-70f2-43d9-b5bf-b5e30b8101a5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.004729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039084-70f2-43d9-b5bf-b5e30b8101a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.005004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e039084-70f2-43d9-b5bf-b5e30b8101a5 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.005639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e039084-70f2-43d9-b5bf-b5e30b8101a5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:20.005993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e039084-70f2-43d9-b5bf-b5e30b8101a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.007924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e039084-70f2-43d9-b5bf-b5e30b8101a5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.015298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 390/773] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Sep 5 16:49:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:20.020561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01f33d-ec39-4223-85ca-550627132a05 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.020844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01f33d-ec39-4223-85ca-550627132a05 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.021265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01f33d-ec39-4223-85ca-550627132a05 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.022608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 391/774] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:20.024951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01f33d-ec39-4223-85ca-550627132a05 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.037058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:20.038278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3476a2-81ae-4aa6-8aa9-fefeba23da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.039229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3476a2-81ae-4aa6-8aa9-fefeba23da05 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.039784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3476a2-81ae-4aa6-8aa9-fefeba23da05 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.040260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3476a2-81ae-4aa6-8aa9-fefeba23da05 None admin] PATH_INFO: `/v3/services/network` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.041016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3476a2-81ae-4aa6-8aa9-fefeba23da05 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.042949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3476a2-81ae-4aa6-8aa9-fefeba23da05 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-aa3476a2-81ae-4aa6-8aa9-fefeba23da05 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:20.054461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:20.055478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:20.056343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:20.064316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 384/775] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:19 2022] GET /identity/v3/services/network => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:20.066286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90bf1e-1508-4e90-b0ed-4ec393c1f00d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.066883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90bf1e-1508-4e90-b0ed-4ec393c1f00d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.067336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90bf1e-1508-4e90-b0ed-4ec393c1f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.087752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4f90bf1e-1508-4e90-b0ed-4ec393c1f00d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:20.089506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4f90bf1e-1508-4e90-b0ed-4ec393c1f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:20.136075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01f33d-ec39-4223-85ca-550627132a05 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.138920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:20.172439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 385/776] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:20.173562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90bf1e-1508-4e90-b0ed-4ec393c1f00d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.183761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.184397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.184671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.184925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.185616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] RBAC: Adding query filter params (interface=public, service_id=a4606c20cc9c491f8e43444d9604491f, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:20.185916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.187276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.200038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 392/777] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/endpoints?service_id=a4606c20cc9c491f8e43444d9604491f&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:20.206610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.328446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:20.354842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad48dd3-38a3-442a-a968-6208b05b1a1f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.378073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90bf1e-1508-4e90-b0ed-4ec393c1f00d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.398868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.399481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.399755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.400120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.400755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.402106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.408084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 393/778] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:20.432476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.433248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.433621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.433985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.434711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] RBAC: Adding query filter params (name=network) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:20.435104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.436629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.442740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 386/779] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:20.452417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.545370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.572254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e61140-aefe-4c92-8aa9-def771ab5b15 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.612220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861e67-5858-4019-9731-40373b0308db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.613028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861e67-5858-4019-9731-40373b0308db None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.613307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861e67-5858-4019-9731-40373b0308db None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.613576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45861e67-5858-4019-9731-40373b0308db None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.614226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45861e67-5858-4019-9731-40373b0308db None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:20.614525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45861e67-5858-4019-9731-40373b0308db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.616919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45861e67-5858-4019-9731-40373b0308db None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.617623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:20.624680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 394/780] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:20.627981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861e67-5858-4019-9731-40373b0308db None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.637426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:20.709886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745268e-ebe4-4ba1-b221-fca80de588be None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.744831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45861e67-5858-4019-9731-40373b0308db None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.762193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.763144 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.763629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.764068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.784380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:20.789499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:20.790387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:20.791232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:20.795937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 387/781] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:20.801818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.814006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.814935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.815411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.815814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.816687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] RBAC: Adding query filter params (interface=public, service_id=34681fac19c9408f97d3a22d8986ff13, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:20.817148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.818658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.827691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 395/782] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/endpoints?service_id=34681fac19c9408f97d3a22d8986ff13&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:20.833272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:20.938423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374291ed-e686-4d62-a976-9d554ab69499 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:20.949664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:20.969532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:20.975093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:20.975718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:20.975998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:20.976300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:20.976959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:20.977261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:20.978628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:20.988118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 388/783] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:20.993465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:21.027222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6573064-359c-4fc3-8734-d8ae4bb3e621 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:21.088843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:21.089495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.089778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.090052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.090716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:21.092281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:21.107727 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 396/784] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/services => generated 772 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:21.142415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58ed64-6141-4e1d-a98d-7e2ae5a3f4d2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:21.180973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bd684-056a-4dab-aa20-59aa8db1bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:21.182028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd684-056a-4dab-aa20-59aa8db1bd7f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.182500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd684-056a-4dab-aa20-59aa8db1bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.182959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd684-056a-4dab-aa20-59aa8db1bd7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.196722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bd684-056a-4dab-aa20-59aa8db1bd7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:21.198584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bd684-056a-4dab-aa20-59aa8db1bd7f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-529bd684-056a-4dab-aa20-59aa8db1bd7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:21.215070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:21.216085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:21.216991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:21.216991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:21.216991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:21.219267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 389/785] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:21.225218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:21.360979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae7386-a35a-40f5-ab22-2336414812f5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:21.407064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:21.421251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:21.422183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.422656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.423177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.424010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:21.424567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:21.426265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:21.442943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 397/786] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:21.444893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42d5d0-0eb8-4fbe-8e07-3d4a0a4e68de None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.445415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42d5d0-0eb8-4fbe-8e07-3d4a0a4e68de None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.445693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42d5d0-0eb8-4fbe-8e07-3d4a0a4e68de None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.447172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 390/787] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:21.448492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:21.452514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82d428-79b8-4ce0-a6d3-f0ae884bfcc2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.452796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82d428-79b8-4ce0-a6d3-f0ae884bfcc2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.453070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82d428-79b8-4ce0-a6d3-f0ae884bfcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.486789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fa82d428-79b8-4ce0-a6d3-f0ae884bfcc2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:21.488082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fa82d428-79b8-4ce0-a6d3-f0ae884bfcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:21.512612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:21.579893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 391/788] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:21.598220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f426c8-0fc2-439c-a0e8-892e32b2f1c3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:21.616541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffad6d-9079-49d4-8df6-2abeed021da5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.616936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffad6d-9079-49d4-8df6-2abeed021da5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.617208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffad6d-9079-49d4-8df6-2abeed021da5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.618887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 392/789] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:21.624906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bf079-387b-45e5-9bf2-9c7ffb1537aa None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.625184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bf079-387b-45e5-9bf2-9c7ffb1537aa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.625451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bf079-387b-45e5-9bf2-9c7ffb1537aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.655602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-537bf079-387b-45e5-9bf2-9c7ffb1537aa None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:21.656834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-537bf079-387b-45e5-9bf2-9c7ffb1537aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:21.673609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:21.674246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.674523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.675144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.686694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:21.688140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:21.710306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:21.711361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:21.712419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 398/790] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:21.720545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:21.759622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 393/791] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:21.816516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:21.895701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c0469-0308-4e71-af71-d766a5c10129 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:21.902969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bf079-387b-45e5-9bf2-9c7ffb1537aa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:21.954463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:21.955646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:21.957583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:21.958820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:21.959745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:21.961093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:21.962859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:21.996580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 399/792] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Sep 5 16:49:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:21.997795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.023589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.048926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bf079-387b-45e5-9bf2-9c7ffb1537aa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.085088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.085732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.086008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.086279 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.096778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:22.101748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:22.102675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 394/793] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:22.103941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.163804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a53dba-21ab-4343-a58a-2eb485596abc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.166811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.209814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.210537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.211044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.211376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.213225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce766574-7d34-47e8-9bc0-dedffd41fc03 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.223403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.228068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:22.248571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:22.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:22.250805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:22.250805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:22.250805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:22.250805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 400/794] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:22.250805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.258827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.259680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.260124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.260574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.264204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.274333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:22.279290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:22.280482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:22.281511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 395/795] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:22.282231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.382536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d503045-3e42-4772-9051-05feca4434bf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.404436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ba20c-2089-48a1-a209-6a98cd4186d7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.418583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.419196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.419475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.419745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.420544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:22.420848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.422325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.436337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 401/796] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:22.437384 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.443181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.443879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.444249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.444587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.445297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:22.445714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.447215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.459754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 396/797] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:22.460311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.460790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.522167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.554001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aa248-39c6-4187-bad6-a2bbaf7520b0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.568800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c6de0d-0b0a-42fa-9210-b2ffba84dfb1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.591588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.592228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.592649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.592937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.593661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:22.593963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.595426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.603788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.604517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 402/798] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:22.604875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.605157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.605733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.605894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.617674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.619419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:22.634320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:22.635339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:22.636232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:22.636232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:22.636232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:22.636232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 397/799] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:22.636895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.696271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.717538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d12558-d144-4ef1-9764-00b3f6685d9e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.741137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e1d83-1213-49ac-82eb-235d22dfd172 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.751019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.755422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.756986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.757823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.758359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.767748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.770132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.778493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.779096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.779373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.779643 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:22.785072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:22.786498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:22.787604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:22.794505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 403/800] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:22.794505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.798607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.800352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:22.817306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:22.818492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:22.819564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:22.819564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:22.819564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:22.820777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 398/801] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:22.821761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.911634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e5a4e-241f-45f9-b6d5-0dd6f820e793 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.927926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed854b-ae27-4809-83fb-a4d0fb55009f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:22.947825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.948467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.948744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.949013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.949647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:22.949947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.951313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.961258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:22.962927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda082ab-5550-43e5-ae7b-9fa343bb0ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:22.964160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda082ab-5550-43e5-ae7b-9fa343bb0ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.964542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda082ab-5550-43e5-ae7b-9fa343bb0ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.964821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda082ab-5550-43e5-ae7b-9fa343bb0ff8 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.965564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda082ab-5550-43e5-ae7b-9fa343bb0ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:22.965871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda082ab-5550-43e5-ae7b-9fa343bb0ff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:22.967381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda082ab-5550-43e5-ae7b-9fa343bb0ff8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:22.971096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 404/802] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:22.972429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:22.980242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 399/803] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:22.981902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df563-5187-4d13-9844-66445fed8f60 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:22.982183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df563-5187-4d13-9844-66445fed8f60 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:22.982450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df563-5187-4d13-9844-66445fed8f60 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:22.986104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 400/804] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:22.986104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609df563-5187-4d13-9844-66445fed8f60 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.008917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:23.076835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1543e8-8823-4495-bd08-2183aa99aeec None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.094640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609df563-5187-4d13-9844-66445fed8f60 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.112815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.113421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.113695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.113957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.114590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:23.114982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.116259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:23.132345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 405/805] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:23.132345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.137973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6288af9-9596-4985-b7be-2d71924a8d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.138690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6288af9-9596-4985-b7be-2d71924a8d1b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.139037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6288af9-9596-4985-b7be-2d71924a8d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.139374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6288af9-9596-4985-b7be-2d71924a8d1b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.146919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6288af9-9596-4985-b7be-2d71924a8d1b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.148511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6288af9-9596-4985-b7be-2d71924a8d1b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d6288af9-9596-4985-b7be-2d71924a8d1b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:23.161014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:23.161925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:23.162840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 401/806] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:49:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:23.163408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58446863-4111-461b-b932-f4e0c3ff7d31 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.163483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58446863-4111-461b-b932-f4e0c3ff7d31 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.163912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58446863-4111-461b-b932-f4e0c3ff7d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.201746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:23.250805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fc7ea-05f1-45ef-a022-5269b8d0d43a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.289484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.290159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.290434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.290704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2756036297064cc19d5fd5d1861703c3/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.293186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=2756036297064cc19d5fd5d1861703c3, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.294639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:23.297978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-58446863-4111-461b-b932-f4e0c3ff7d31 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:23.299154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-58446863-4111-461b-b932-f4e0c3ff7d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:23.308856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 406/807] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:23 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2756036297064cc19d5fd5d1861703c3/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:23.317259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.383217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 402/808] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:23.384104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58446863-4111-461b-b932-f4e0c3ff7d31 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.405445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:23.464990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4a0a6-8db5-4741-813d-c3d062de264d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.493160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:23.511192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8064b-8179-44c3-9522-07be8c4ca8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.516181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8064b-8179-44c3-9522-07be8c4ca8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.516181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8064b-8179-44c3-9522-07be8c4ca8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.516181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8064b-8179-44c3-9522-07be8c4ca8f8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.521574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8064b-8179-44c3-9522-07be8c4ca8f8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.523117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8064b-8179-44c3-9522-07be8c4ca8f8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:23.524597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58446863-4111-461b-b932-f4e0c3ff7d31 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b3b8064b-8179-44c3-9522-07be8c4ca8f8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:23.535553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:23.536725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:23.537862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:23.540404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 407/809] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:49:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:23.541119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509acc83-0401-45d3-8e33-6847a71e6be1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.541484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509acc83-0401-45d3-8e33-6847a71e6be1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.541841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509acc83-0401-45d3-8e33-6847a71e6be1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.543491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 408/810] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:23.549010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509acc83-0401-45d3-8e33-6847a71e6be1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.568543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0921aa-163e-4d8f-95fc-3f23090f51e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.569171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0921aa-163e-4d8f-95fc-3f23090f51e6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.569456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0921aa-163e-4d8f-95fc-3f23090f51e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.569737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0921aa-163e-4d8f-95fc-3f23090f51e6 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.570732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0921aa-163e-4d8f-95fc-3f23090f51e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:23.571048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0921aa-163e-4d8f-95fc-3f23090f51e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.572945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0921aa-163e-4d8f-95fc-3f23090f51e6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:23.600175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 403/811] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:49:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:23.600175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5344e-82b7-4526-92d1-158e07b76768 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.600175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5344e-82b7-4526-92d1-158e07b76768 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.600175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5344e-82b7-4526-92d1-158e07b76768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.622161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dbc5344e-82b7-4526-92d1-158e07b76768 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:23.623772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dbc5344e-82b7-4526-92d1-158e07b76768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:23.665829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509acc83-0401-45d3-8e33-6847a71e6be1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.695741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:23.702479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 404/812] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:23.703272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5344e-82b7-4526-92d1-158e07b76768 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.704553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.705388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.706159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.707567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.708497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:23.709009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.710625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:23.741499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 409/813] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Sep 5 16:49:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:23.742309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.786771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:23.831033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5344e-82b7-4526-92d1-158e07b76768 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.856079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f650-fe06-4fb9-8123-656e33e65bdd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:23.865630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.866545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.866959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.867386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.897277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:23.898203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:23.898482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:23.898751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:23.899606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.901639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:23.913242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:23.914963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:23.915976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 410/814] 10.222.0.23 () {56 vars in 1129 bytes} [Mon Sep 5 16:49:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:23.916915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:23.960140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:23.966856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:23.968361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.038685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e70cd89-cc01-402f-b203-7806d7dc86d8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.040045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:24.074885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 405/815] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:24.075962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.082910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.083555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.083832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.084104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.096187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.098116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:24.113328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:24.114327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:24.115241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:24.115241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:24.115241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:24.119154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 411/816] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:24.119979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.179931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:24.184327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1ad16-0917-4465-a31b-cf903a921f54 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.220597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.221527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.221989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.222421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.223193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:24.223641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.224567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145cbe5-e75f-4fdb-b1d6-1018a450b4c5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.225362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.230992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 406/817] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:24.231979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.266102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.266733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.267013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.267286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] PATH_INFO: `/v3/services/network` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.267862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.269349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:24.278238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:24.279441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:24.280414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 412/818] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:24.281209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.303977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:24.348172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99597d-8b7c-497a-a65d-7f41efc37bf6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.387321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.387968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.388281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.388559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.400652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.402098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.402840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d38d0-7b8d-4933-8a64-6448035fedcb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:24.417145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:24.418141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:24.419031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:24.419031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:24.419031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:24.419031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 407/819] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:24.419768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.436588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:24.452189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.452860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.453136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.453620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.454282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:24.454591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.456601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.468751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 413/820] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:24.469577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.536965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ff124-bd90-4b1e-81a3-2aff7e2d645f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.556334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:24.572009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.572651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.572935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.573210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.573840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:24.574146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.575538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.582186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 408/821] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:24.582765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c502e0b-7269-4b54-9c80-63874414f3f5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.583005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.624048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.625126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.625395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.625660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.626277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:24.626574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.628083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.641927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 414/822] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:24.641927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.689371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:24.717061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02504e8b-f367-4a8e-8120-8b3ca7e9270d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.764615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ed67f-67da-4fa6-af3b-5def10da78e9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.773879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.774816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.775269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.775722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.776410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:24.776701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.778082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.799492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 409/823] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:24.800930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.802635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.803278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.803560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.803833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.811291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.812889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:24.825272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:24.827535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 415/824] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:24.829600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:24.830219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:24.946916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d33e91-00f9-4c11-9da0-67a92b31041c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.953456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b98c8-9f17-4532-995c-c89126b37768 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:24.992217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:24.992863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:24.993139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:24.993403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:24.994092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:24.995458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.000840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:25.002074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.002488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.003272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.004093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:25.004592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:25.005294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:25.008080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.019209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 416/825] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:25.020414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:25.032571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 410/826] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:25.033455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:25.110012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:25.165372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0b0c0-21cf-41ee-8c7a-f1953eb9bf3f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:25.196314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0aae68-09e7-41ca-8bff-80317e68825d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:25.215213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:25.216068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.216397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.216673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.226551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:25.228562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:25.246762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:25.247878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:25.248990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:25.261641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 417/827] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:25.261641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:25.262342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:25.262991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.263266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.263533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.264317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:25.264637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:25.266232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.283453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 411/828] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:25.285823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:25.354416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:25.400064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f23a3f-05ff-4cec-ad17-56aca9bbfa5b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:25.421250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3b2af-5b86-43bc-b52a-567681b278ab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:25.439112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:25.450907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95480df3-ee87-4e41-b58d-2dc885c7ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:25.451513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95480df3-ee87-4e41-b58d-2dc885c7ba7c None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.451794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95480df3-ee87-4e41-b58d-2dc885c7ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.452055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95480df3-ee87-4e41-b58d-2dc885c7ba7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.452777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95480df3-ee87-4e41-b58d-2dc885c7ba7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:25.455646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95480df3-ee87-4e41-b58d-2dc885c7ba7c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.468261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:25.468470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.468738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.469011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.470245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:25.470245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:25.475862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.493511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 412/829] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:25.493511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:25.495578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 418/830] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:25.496184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc4fc-0640-41b7-a1c8-30eaea381da4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.496364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc4fc-0640-41b7-a1c8-30eaea381da4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.496753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cc4fc-0640-41b7-a1c8-30eaea381da4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.502262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 419/831] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:25.502262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc4fc-0640-41b7-a1c8-30eaea381da4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:25.630831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ea836-d80e-4e03-b456-f3154dcc3c4e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:25.643536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:25.659952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cc4fc-0640-41b7-a1c8-30eaea381da4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:25.671008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2364e6d-9735-4689-b7f9-ee8e474f4b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:25.671688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2364e6d-9735-4689-b7f9-ee8e474f4b63 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.671969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2364e6d-9735-4689-b7f9-ee8e474f4b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.673002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2364e6d-9735-4689-b7f9-ee8e474f4b63 None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2f2310a3da094b959270d797bd2f1553/roles/ba985e88b6d44ba38445fe86872360b2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.692822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2364e6d-9735-4689-b7f9-ee8e474f4b63 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=2f2310a3da094b959270d797bd2f1553, role_id=ba985e88b6d44ba38445fe86872360b2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:25.694590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2364e6d-9735-4689-b7f9-ee8e474f4b63 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.707012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:25.707635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.707909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.708232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.709128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=c30e20ce4d63452095aedb60249f0cce) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:25.709447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:25.710890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:25.713603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 413/832] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:25 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2f2310a3da094b959270d797bd2f1553/roles/ba985e88b6d44ba38445fe86872360b2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:49:25.715121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3606c-a933-4b0b-ba78-d68a88014cf0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:25.715553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3606c-a933-4b0b-ba78-d68a88014cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:25.716009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3606c-a933-4b0b-ba78-d68a88014cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:25.718581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 420/833] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:25 2022] GET /identity/v3/role_assignments?user.id=c30e20ce4d63452095aedb60249f0cce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:25.737717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-20a3606c-a933-4b0b-ba78-d68a88014cf0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:25.739413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-20a3606c-a933-4b0b-ba78-d68a88014cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:25.823178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 414/834] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:26.100058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:26.147364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:26.207492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa78db3-9fd8-403a-b033-2a503020146c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:26.242747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:26.243626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:26.244043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:26.244473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:26.255435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:26.262194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:26.263368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:26.264422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:26.266802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 421/835] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:26.273139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3606c-a933-4b0b-ba78-d68a88014cf0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:26.376660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:26.385316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3606c-a933-4b0b-ba78-d68a88014cf0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:26.420515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:26.421131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:26.421409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:26.421678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:26.422335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:26.422633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:26.424053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:26.432768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 415/836] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:26.438383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:26.541620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271196a-bc62-4acd-81eb-2aa4f31af7b3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:26.543104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:26.580282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:26.581133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:26.581734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:26.581734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:26.591846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:26.593313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:26.608142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:26.609125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:26.610027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:26.610027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:26.610027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:26.610027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 422/837] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:26.615177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:26.723431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c653d2-aab5-4689-bbbf-9c4dc91c73b9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:26.758301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:26.758921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:26.759195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:26.759472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:26.760177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:26.760486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:26.761846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:26.774611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 416/838] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:26.780174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:26.895270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a4431-ac77-41bd-896e-221c71b6499a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:26.933271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:26.933922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:26.934197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:26.934463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:26.943491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:26.944968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:26.957935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:26.959066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:26.960333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 423/839] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:26.966834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:26.983440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:27.071263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b3a35-d6f2-4565-a059-9726c5a3d3f5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:27.105092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630e9fef-2029-4a4d-994c-ae0a87ff54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:27.106228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e9fef-2029-4a4d-994c-ae0a87ff54df None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.106668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e9fef-2029-4a4d-994c-ae0a87ff54df None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.107072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e9fef-2029-4a4d-994c-ae0a87ff54df None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.107929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e9fef-2029-4a4d-994c-ae0a87ff54df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:27.108393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e9fef-2029-4a4d-994c-ae0a87ff54df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:27.109948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630e9fef-2029-4a4d-994c-ae0a87ff54df None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:27.128169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 417/840] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Sep 5 16:49:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:27.134242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:27.151466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:27.160249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d205-19cd-46da-81af-3a0672e2c22b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.161202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d205-19cd-46da-81af-3a0672e2c22b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.161645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d205-19cd-46da-81af-3a0672e2c22b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.163754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 418/841] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:27.168492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884b01-5e0d-404f-9646-05631c42d4bd None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.168957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884b01-5e0d-404f-9646-05631c42d4bd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.169672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0884b01-5e0d-404f-9646-05631c42d4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.192344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d0884b01-5e0d-404f-9646-05631c42d4bd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:27.193754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d0884b01-5e0d-404f-9646-05631c42d4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:27.246592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f6ea8-d132-41e9-81bf-7c0347fe9fe8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:27.277812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 419/842] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:27.278340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:27.289939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:27.290815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.291292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.291713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.360869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:27.362633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:27.405806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:27.464665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 424/843] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:27.470282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884b01-5e0d-404f-9646-05631c42d4bd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:27.551773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:27.583247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0884b01-5e0d-404f-9646-05631c42d4bd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:27.627345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:27.628280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.628705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.628984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.642045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:27.643537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:27.656581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a6a8cc-b588-43c3-9956-8083b0de5d4a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:27.662437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:27.663365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:27.664689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:27.664689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:27.664689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:27.668781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 420/844] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:27.669661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:27.676377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:27.698519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b27b42-a7b0-4428-b27f-8f69e1758844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:27.699496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b27b42-a7b0-4428-b27f-8f69e1758844 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b27b42-a7b0-4428-b27f-8f69e1758844 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.700481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b27b42-a7b0-4428-b27f-8f69e1758844 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.701281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b27b42-a7b0-4428-b27f-8f69e1758844 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:27.702834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b27b42-a7b0-4428-b27f-8f69e1758844 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-60b27b42-a7b0-4428-b27f-8f69e1758844 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:27.712522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:27.713556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:27.714368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:27.715112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 425/845] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:27.720297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed697e-84fd-4576-b6a4-6bfa47bbdd5e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.720297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed697e-84fd-4576-b6a4-6bfa47bbdd5e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.720297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed697e-84fd-4576-b6a4-6bfa47bbdd5e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.721777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 426/846] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:27.722472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed697e-84fd-4576-b6a4-6bfa47bbdd5e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:27.733112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:27.785128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3739-4ca7-49e0-b3c0-a452ac2bbc1f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:27.820283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8244a-5b96-470f-a05d-889960e75480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:27.820810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8244a-5b96-470f-a05d-889960e75480 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.821088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8244a-5b96-470f-a05d-889960e75480 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.821365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8244a-5b96-470f-a05d-889960e75480 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.822016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8244a-5b96-470f-a05d-889960e75480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:27.822319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8244a-5b96-470f-a05d-889960e75480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:27.823695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8244a-5b96-470f-a05d-889960e75480 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:27.828052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed697e-84fd-4576-b6a4-6bfa47bbdd5e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:27.835604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 421/847] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:27.836810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0c5c-c93e-47fe-a21d-6224bdef6a5c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.837093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0c5c-c93e-47fe-a21d-6224bdef6a5c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.837367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f0c5c-c93e-47fe-a21d-6224bdef6a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.854982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e05f0c5c-c93e-47fe-a21d-6224bdef6a5c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:27.856535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e05f0c5c-c93e-47fe-a21d-6224bdef6a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:27.863001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd494d-7910-4ee0-b970-617e37e725da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:27.863607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd494d-7910-4ee0-b970-617e37e725da None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.863883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd494d-7910-4ee0-b970-617e37e725da None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.864173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd494d-7910-4ee0-b970-617e37e725da None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.864794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd494d-7910-4ee0-b970-617e37e725da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:27.865091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd494d-7910-4ee0-b970-617e37e725da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:27.866446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd494d-7910-4ee0-b970-617e37e725da None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:27.871302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 427/848] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:27.872482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826777-d4c1-40c4-9ee4-1983c67f96af None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:27.872758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826777-d4c1-40c4-9ee4-1983c67f96af None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:27.873032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0826777-d4c1-40c4-9ee4-1983c67f96af None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:27.874211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 428/849] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:27.874879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0826777-d4c1-40c4-9ee4-1983c67f96af None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:27.919443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:27.927974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 422/850] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:27.928854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f0c5c-c93e-47fe-a21d-6224bdef6a5c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:27.981182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0826777-d4c1-40c4-9ee4-1983c67f96af None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:27.982613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:28.016905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf695aff-3d4e-4704-8c85-9deed1920392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.017558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf695aff-3d4e-4704-8c85-9deed1920392 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.017844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf695aff-3d4e-4704-8c85-9deed1920392 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.018112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf695aff-3d4e-4704-8c85-9deed1920392 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.024997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf695aff-3d4e-4704-8c85-9deed1920392 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.026427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf695aff-3d4e-4704-8c85-9deed1920392 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bf695aff-3d4e-4704-8c85-9deed1920392 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:28.038064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:28.038933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:28.039903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 429/851] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:28.040417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fbce6-6c5e-4b0c-a2bd-0d399410290b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.041234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fbce6-6c5e-4b0c-a2bd-0d399410290b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.041234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fbce6-6c5e-4b0c-a2bd-0d399410290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.046666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f0c5c-c93e-47fe-a21d-6224bdef6a5c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.058909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ab5fbce6-6c5e-4b0c-a2bd-0d399410290b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:28.060088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ab5fbce6-6c5e-4b0c-a2bd-0d399410290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:28.085957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.086569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.086843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.087106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.087725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:28.088022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.089423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.094103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 423/852] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:28.094691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.154816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 430/853] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:28.192416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fbce6-6c5e-4b0c-a2bd-0d399410290b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.208633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f9cb2-38b9-4ae8-9a91-958586f47871 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.217475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:28.240410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:28.251267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.252122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.252497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.252810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.253530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:28.254048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.255603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.270711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 424/854] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:28.277659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.331188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fbce6-6c5e-4b0c-a2bd-0d399410290b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.386906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.387618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.388061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.388489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.389134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.390782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.402821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2450ab6-4717-4661-bde1-f19517e414d6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:28.403323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:28.404257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:28.405165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 431/855] 10.222.0.23 () {56 vars in 1132 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:28.416381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.451105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.451772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.452053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.452403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.453132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2756036297064cc19d5fd5d1861703c3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:28.453490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.454962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.461776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 425/856] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/role_assignments?user.id=2756036297064cc19d5fd5d1861703c3&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:28.462571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.508337 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:28.543598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:28.552306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45855fd-4fc2-42d3-9699-e07a5ff15019 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.597351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.598134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.598617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.598977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.600106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:28.600426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.601845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.611657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbd6aa-4511-4294-baf8-6fb277ed4dce None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.612596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 432/857] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:28.613949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.654880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.655449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.655726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.655997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.668233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:28.673849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:28.674805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:28.675649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 426/858] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:28.680683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.738293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e4dd5-f9e1-412d-ad9f-164c9081d212 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.775081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.775700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.775979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.776297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.776933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:28.777235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.778618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.783876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 433/859] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:28.813326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153a70d-1648-48e6-ac67-7555f93587cd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:28.816496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:28.853662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:28.854306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:28.854631 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:28.854911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:28.855895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:28.856222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:28.858018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:28.868056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 427/860] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:28.873589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:28.903809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:28.989649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09a292-a305-4235-a118-5b11edeee021 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:29.026942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:29.027582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:29.027861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:29.028134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:29.040874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:29.042326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:29.060006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:29.061015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:29.061974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:29.061974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:29.061974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:29.062142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 434/861] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:29.068518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:29.177770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54f95a-84a1-42d7-a8fa-5bca5832f627 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:29.212614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:29.213486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:29.213950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:29.214382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:29.215169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:29.215617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:29.217170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:29.231508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 428/862] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:29.237269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:29.318441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:29.344212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb6480-b839-4840-9806-1188f23684f9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:29.385387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:29.386275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:29.386752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:29.387174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:29.396693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:29.399173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:29.413604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:29.414788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:29.415979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:29.418300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 435/863] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:49:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:29.425602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:29.439157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:29.528278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368d89f-7c4a-4b6d-baec-adc6bbdb1e68 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:29.563556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:29.564190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:29.564475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:29.564822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:29.565646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:29.566361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:29.567374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:29.585828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 429/864] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Sep 5 16:49:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:29.591464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:29.704068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7686387-1dd0-4543-91cc-977a6e3d4352 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:29.739634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a783b4c-c0ef-43da-959b-686d51fdcd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:29.740742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783b4c-c0ef-43da-959b-686d51fdcd30 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:29.741214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783b4c-c0ef-43da-959b-686d51fdcd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:29.741681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783b4c-c0ef-43da-959b-686d51fdcd30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:29.745336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:29.818103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a783b4c-c0ef-43da-959b-686d51fdcd30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:29.819603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a783b4c-c0ef-43da-959b-686d51fdcd30 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:29.930805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 436/865] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:29.932103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:30.011703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:30.043831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264dbaa-fbbd-4831-82b6-bc7365df324f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:30.058689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581735b-a0cf-4ca4-97ef-1dcbbbb9ffb9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.059045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581735b-a0cf-4ca4-97ef-1dcbbbb9ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.059228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581735b-a0cf-4ca4-97ef-1dcbbbb9ffb9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.060997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 437/866] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:30.066844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff247a-64a0-4a59-b654-4b34b9712d83 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.067181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff247a-64a0-4a59-b654-4b34b9712d83 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.067381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff247a-64a0-4a59-b654-4b34b9712d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.086643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-78ff247a-64a0-4a59-b654-4b34b9712d83 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:30.087819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-78ff247a-64a0-4a59-b654-4b34b9712d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:30.088752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:30.089646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.090121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.090587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.101199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:30.102923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:30.118865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:30.119830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:30.120791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:30.120791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:30.120791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:30.121688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:30.123672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 430/867] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:30.130364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:30.180174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 438/868] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:30.236803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8629e5e-597a-4313-9908-10d7d4ec8c9e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:30.260515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:30.272436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:30.273061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.273389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.273670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.274313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:30.274617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:30.276002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:30.289033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 431/869] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:30.293973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff247a-64a0-4a59-b654-4b34b9712d83 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:30.396391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff247a-64a0-4a59-b654-4b34b9712d83 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:30.436322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:30.437355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.437835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.438304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.445794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:30.447504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:30.456134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:30.460442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:30.461319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:30.465016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 439/870] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:30.469061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:30.501265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:30.578842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ecf65-d7bd-4e7e-9388-03db7f436961 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:30.600299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cdf36-47c0-4380-8424-40990cd0691b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:30.616275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d29dc1-7542-4ec2-936b-20b18cb5e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:30.617282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d29dc1-7542-4ec2-936b-20b18cb5e46d None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.618046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d29dc1-7542-4ec2-936b-20b18cb5e46d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.618491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d29dc1-7542-4ec2-936b-20b18cb5e46d None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.619320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d29dc1-7542-4ec2-936b-20b18cb5e46d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:30.621451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d29dc1-7542-4ec2-936b-20b18cb5e46d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:30.635353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:30.635960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.636268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.636544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.637245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:30.637553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:30.639002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:30.648196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 432/871] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:30 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:49:30.648196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f81fd-f1c2-4a93-9400-854fc567c461 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.648196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f81fd-f1c2-4a93-9400-854fc567c461 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.648196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f81fd-f1c2-4a93-9400-854fc567c461 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.649086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 433/872] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:30.654521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f522d-238f-4734-a763-31bc80ced99f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.654821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f522d-238f-4734-a763-31bc80ced99f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.655076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f522d-238f-4734-a763-31bc80ced99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.657111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 440/873] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:30.663450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:30.687213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ab0f522d-238f-4734-a763-31bc80ced99f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:30.688577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ab0f522d-238f-4734-a763-31bc80ced99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:30.731814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:30.771245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:30.789840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc98ed-b7a6-4cd8-9d64-fe82c0aa1ee6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:30.796391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 434/874] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:30.840385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282f1d2-4719-411a-99e1-e4ba01f5285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:30.841001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282f1d2-4719-411a-99e1-e4ba01f5285f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.841281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282f1d2-4719-411a-99e1-e4ba01f5285f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.841555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282f1d2-4719-411a-99e1-e4ba01f5285f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.842292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282f1d2-4719-411a-99e1-e4ba01f5285f None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2f2310a3da094b959270d797bd2f1553) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:30.842601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282f1d2-4719-411a-99e1-e4ba01f5285f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:30.844042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282f1d2-4719-411a-99e1-e4ba01f5285f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:30.850781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 441/875] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:30 2022] GET /identity/v3/role_assignments?user.id=2f2310a3da094b959270d797bd2f1553&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:30.902727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989fe13-f43e-4c11-ab09-1c7f24d6f958 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.903063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989fe13-f43e-4c11-ab09-1c7f24d6f958 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.903280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989fe13-f43e-4c11-ab09-1c7f24d6f958 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.905106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 435/876] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:30.911286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd46fee-f08c-44cb-a131-6b1d0b0c3871 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:30.911643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd46fee-f08c-44cb-a131-6b1d0b0c3871 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:30.911849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd46fee-f08c-44cb-a131-6b1d0b0c3871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:30.931430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dbd46fee-f08c-44cb-a131-6b1d0b0c3871 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:30.932697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dbd46fee-f08c-44cb-a131-6b1d0b0c3871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:31.007804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 442/877] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:31.068650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989fe13-f43e-4c11-ab09-1c7f24d6f958 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:31.190855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989fe13-f43e-4c11-ab09-1c7f24d6f958 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:31.227528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:31.228178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:31.228460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:31.228733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:31.229352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:31.231091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:31.240561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:31.241572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:31.242392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:31.246619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 436/878] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:31.247954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd46fee-f08c-44cb-a131-6b1d0b0c3871 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:31.269943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:31.306111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:31.342317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:31.368493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd46fee-f08c-44cb-a131-6b1d0b0c3871 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:31.395683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62866b-9050-42a4-bf67-3f51b71f93bd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:31.413341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:31.413983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:31.414356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:31.414631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:31.415316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:31.415624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:31.417020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:31.422334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 443/879] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:31.428822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:31.436143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1f637-b741-4bd7-b5bd-514c13e93b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:31.436801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f637-b741-4bd7-b5bd-514c13e93b4e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:31.437078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f637-b741-4bd7-b5bd-514c13e93b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:31.437347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1f637-b741-4bd7-b5bd-514c13e93b4e None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:31.438063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1f637-b741-4bd7-b5bd-514c13e93b4e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:31.439617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1f637-b741-4bd7-b5bd-514c13e93b4e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:31.461066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 437/880] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:31 2022] POST /identity/v3/services => generated 269 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:49:31.557222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe697e-84e2-468f-a4e6-414800299bd0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:31.602874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe85c8d-6101-426e-be95-e145781214c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:31.603413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85c8d-6101-426e-be95-e145781214c9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:31.603683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85c8d-6101-426e-be95-e145781214c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:31.603946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe85c8d-6101-426e-be95-e145781214c9 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:31.604683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe85c8d-6101-426e-be95-e145781214c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:31.604980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe85c8d-6101-426e-be95-e145781214c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:31.606638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe85c8d-6101-426e-be95-e145781214c9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:31.612316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 444/881] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:32.659604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f41e-8605-4963-b24e-5d968938f9d3 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:32.660308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f41e-8605-4963-b24e-5d968938f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:32.660308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f41e-8605-4963-b24e-5d968938f9d3 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:32.661678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 438/882] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:32.667145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6432f-fe8c-4abb-93ae-21873c58264f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:32.667981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6432f-fe8c-4abb-93ae-21873c58264f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:32.668480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6432f-fe8c-4abb-93ae-21873c58264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:32.676066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:32.687666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2ec6432f-fe8c-4abb-93ae-21873c58264f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:32.689142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2ec6432f-fe8c-4abb-93ae-21873c58264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:32.761105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 445/883] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:32.851868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9481718-7e6d-4b9a-835f-a078be3c0221 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:32.852521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9481718-7e6d-4b9a-835f-a078be3c0221 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:32.853000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9481718-7e6d-4b9a-835f-a078be3c0221 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:32.854790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 439/884] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:32.860454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72ff1-8dcf-4b8c-91cb-0371c13d6ba5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:32.861001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72ff1-8dcf-4b8c-91cb-0371c13d6ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:32.861476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72ff1-8dcf-4b8c-91cb-0371c13d6ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:32.879938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6ee72ff1-8dcf-4b8c-91cb-0371c13d6ba5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:32.881458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6ee72ff1-8dcf-4b8c-91cb-0371c13d6ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:32.949797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 446/885] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:33.035712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9481718-7e6d-4b9a-835f-a078be3c0221 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.130920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:33.144688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9481718-7e6d-4b9a-835f-a078be3c0221 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.179506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.180143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.180442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.180714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.181291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:33.182650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:33.191962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:33.193020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:33.193856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 440/886] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:33.201263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee72ff1-8dcf-4b8c-91cb-0371c13d6ba5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.217803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.271132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:33.316285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee72ff1-8dcf-4b8c-91cb-0371c13d6ba5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.350824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84985428-8f23-4479-be82-8846ed2ccf2e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.352862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.353581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.353862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.354134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.354770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:33.355076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:33.356797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:33.362214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 447/887] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:33.367805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.394215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9cbc-7d07-428f-a1c1-75d806081f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.394860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9cbc-7d07-428f-a1c1-75d806081f02 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.395141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9cbc-7d07-428f-a1c1-75d806081f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.395409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9cbc-7d07-428f-a1c1-75d806081f02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.406234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9cbc-7d07-428f-a1c1-75d806081f02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d4bf9cbc-7d07-428f-a1c1-75d806081f02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:33.411179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:33.412118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:33.413263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:33.416306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 441/888] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:33.417728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7faa7d-1751-4b53-8389-5becbf572bb0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.418012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7faa7d-1751-4b53-8389-5becbf572bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.418285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7faa7d-1751-4b53-8389-5becbf572bb0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.421085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 442/889] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:33.421646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7faa7d-1751-4b53-8389-5becbf572bb0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.443136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:33.484405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539eeb0-a7cc-440e-a680-b0ae3f47dae1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.513777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:33.525291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359245a-96b2-464e-99ad-dade729bbc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.525911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359245a-96b2-464e-99ad-dade729bbc4e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.526192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359245a-96b2-464e-99ad-dade729bbc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.526460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359245a-96b2-464e-99ad-dade729bbc4e None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.527116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359245a-96b2-464e-99ad-dade729bbc4e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:33.527460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359245a-96b2-464e-99ad-dade729bbc4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:33.528853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359245a-96b2-464e-99ad-dade729bbc4e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:33.534485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 448/890] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:33.535658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b10f-8186-4b49-b057-2f9177211d86 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.535941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b10f-8186-4b49-b057-2f9177211d86 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.536246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b10f-8186-4b49-b057-2f9177211d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.545758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7faa7d-1751-4b53-8389-5becbf572bb0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.554547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9168b10f-8186-4b49-b057-2f9177211d86 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:33.555739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9168b10f-8186-4b49-b057-2f9177211d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:33.585611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.586226 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.586507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.586781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.587447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:33.587758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:33.589246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:33.597543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 443/891] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:33.598349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.640090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 449/892] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:33.640090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168b10f-8186-4b49-b057-2f9177211d86 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.692376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:33.715927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbff64a-3d1e-485d-a575-cfdbad011906 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.746280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168b10f-8186-4b49-b057-2f9177211d86 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.751398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75428ece-d02d-4197-b0bc-33395c21042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.752004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75428ece-d02d-4197-b0bc-33395c21042e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.752312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75428ece-d02d-4197-b0bc-33395c21042e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.752699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75428ece-d02d-4197-b0bc-33395c21042e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.753389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75428ece-d02d-4197-b0bc-33395c21042e None admin] RBAC: Adding query filter params (interface=public, service_id=8f446ab538af41f4846dfcca64563bf2, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:33.753692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75428ece-d02d-4197-b0bc-33395c21042e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:33.755060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75428ece-d02d-4197-b0bc-33395c21042e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:33.762203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 444/893] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/endpoints?service_id=8f446ab538af41f4846dfcca64563bf2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:33.763433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e2c21-fc09-48cf-9553-8771ce873431 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.763712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e2c21-fc09-48cf-9553-8771ce873431 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.763982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e2c21-fc09-48cf-9553-8771ce873431 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.765360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 445/894] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:33.770216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e2c21-fc09-48cf-9553-8771ce873431 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.777069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:33.791225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033a5b7-fdcf-432e-9aa9-b55a69430ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.792212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a5b7-fdcf-432e-9aa9-b55a69430ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.792632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a5b7-fdcf-432e-9aa9-b55a69430ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.793042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033a5b7-fdcf-432e-9aa9-b55a69430ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.804885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033a5b7-fdcf-432e-9aa9-b55a69430ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:33.806786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033a5b7-fdcf-432e-9aa9-b55a69430ed5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0033a5b7-fdcf-432e-9aa9-b55a69430ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:33.823172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:33.824344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:33.825405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:33.825405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:33.825405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:33.826296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 450/895] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:33.827492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702cf61-78db-4c77-b33f-2fb4a4f466d2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.827774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702cf61-78db-4c77-b33f-2fb4a4f466d2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.828040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702cf61-78db-4c77-b33f-2fb4a4f466d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.847280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2702cf61-78db-4c77-b33f-2fb4a4f466d2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:33.848512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2702cf61-78db-4c77-b33f-2fb4a4f466d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:33.877255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e2c21-fc09-48cf-9553-8771ce873431 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:33.913509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:33.914124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:33.914406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:33.914678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:33.915280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:33.916706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:33.916924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 451/896] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:33.924722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702cf61-78db-4c77-b33f-2fb4a4f466d2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.924831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 446/897] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:49:33.924831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:33.954164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.033464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702cf61-78db-4c77-b33f-2fb4a4f466d2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.036950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.072721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.073649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.074061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.074435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.075192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:34.075570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.077063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.087147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6866082-64d1-400a-8fb4-fd592a331393 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.092142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 452/898] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:34.097980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.129753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.130386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.130671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.130946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.131527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.134526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:34.143847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:34.144841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:34.146457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 447/899] 10.222.0.23 () {56 vars in 1132 bytes} [Mon Sep 5 16:49:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:34.150754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.222571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a172af6-68a9-43c4-b7dd-4d6ac0c88009 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.239715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.258682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.259311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.259998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.260304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.268491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.270067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.272052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc12cc-d08d-4681-86db-26b737c03127 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:34.281343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:34.282304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 453/900] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:34.283266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.310518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.314512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.315344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.315606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.315900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.316574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:34.316876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.318252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.328295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 448/901] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:34.329875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.391337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc64935-bf03-432c-bd83-2501fbf6f0ae None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.427593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.428235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.428532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.428773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.429574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.430871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.436292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ecd1d-146e-49b8-bda9-83dca8107617 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.449144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 454/902] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:34 2022] POST /identity/v3/services => generated 241 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:34.450031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.501463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.511342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.512666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.512781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.514182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.514900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:34.515228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.516749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.543899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 449/903] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:34.544931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.565197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.591313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b21e4-d9e5-4197-9b2c-57d62d3af72c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.639297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.639928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.640244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.640515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.641494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:34.641801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.643298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.649012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 455/904] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:34.654905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.672563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e196dc-a188-4a45-83e5-d7a1f0466d2f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.722860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.724505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.724964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.725635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.782082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1473a-8ee8-4509-9330-53ef7eecd806 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:34.800670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.817825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.819832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.833525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:34.834745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:34.835122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:34.835493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:34.836603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] RBAC: Adding query filter params (interface=public, service_id=8cadf27af48c4b6897788497f3aa54d2, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:34.837014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:34.838749 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:34.848230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 456/905] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/endpoints?service_id=8cadf27af48c4b6897788497f3aa54d2&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:34.851676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.875197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:34.948593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 450/906] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:34.950565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:34.966992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756c847-44a0-496c-ae9c-0dbb0b191ad7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:35.002715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:35.003405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.003689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.003965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.004624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:35.006059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:35.017477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 457/907] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:49:35.062936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:35.076980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbd770-619b-47f9-b642-2e2b53f284de None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:35.116549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:35.117275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.117561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.117836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.129822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:35.131320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:35.148128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:35.149170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:35.150042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:35.150042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:35.150042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:35.150042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 451/908] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:35.156552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:35.238744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:35.263263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f287d1-79f9-4796-9c0e-753479f99742 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:35.298498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:35.299163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.299496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.299771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.300444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:35.300749 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:35.302114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:35.314845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 458/909] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:35.320591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:35.438622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b01f9-e410-4324-97dd-d5ea7ac04e88 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:35.474094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:35.474992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.475413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.475843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.483503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:35.485255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:35.488093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:35.497784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:35.498880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:35.499985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:35.502394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 452/910] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:35.508830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:35.623293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac6e98-ced1-4eb2-9726-b2ddaf95455a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:35.658343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a9bac-6a36-4f0e-bdd5-410ea6545007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:35.658996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a9bac-6a36-4f0e-bdd5-410ea6545007 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.659341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a9bac-6a36-4f0e-bdd5-410ea6545007 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.659673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a9bac-6a36-4f0e-bdd5-410ea6545007 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.660464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a9bac-6a36-4f0e-bdd5-410ea6545007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:35.660829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a9bac-6a36-4f0e-bdd5-410ea6545007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:35.662312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a9bac-6a36-4f0e-bdd5-410ea6545007 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:35.664913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:35.675966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 459/911] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:35.682117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:35.785724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad7130-9ca7-4420-bea9-e2f49a55af06 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:35.820074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774497e5-c02f-4ab4-9c96-b15b0c50f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:35.820725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774497e5-c02f-4ab4-9c96-b15b0c50f5bd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.821004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774497e5-c02f-4ab4-9c96-b15b0c50f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.821281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774497e5-c02f-4ab4-9c96-b15b0c50f5bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.822007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774497e5-c02f-4ab4-9c96-b15b0c50f5bd None admin] RBAC: Adding query filter params (role.id=59ee14375b7b400983f97480ca2fe124, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2f2310a3da094b959270d797bd2f1553) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:35.822317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774497e5-c02f-4ab4-9c96-b15b0c50f5bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:35.823770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774497e5-c02f-4ab4-9c96-b15b0c50f5bd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:35.829946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 453/912] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:35 2022] GET /identity/v3/role_assignments?user.id=2f2310a3da094b959270d797bd2f1553&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=59ee14375b7b400983f97480ca2fe124 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:35.925618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed645cf-22d5-4518-9155-0de1f5527037 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.926132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed645cf-22d5-4518-9155-0de1f5527037 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.926132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed645cf-22d5-4518-9155-0de1f5527037 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.927613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 460/913] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:35.933427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1c783-e916-474f-9d0e-aa7fe3c84883 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:35.933735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1c783-e916-474f-9d0e-aa7fe3c84883 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:35.933971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1c783-e916-474f-9d0e-aa7fe3c84883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:35.953743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-22e1c783-e916-474f-9d0e-aa7fe3c84883 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:35.954962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-22e1c783-e916-474f-9d0e-aa7fe3c84883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:36.020934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:36.030639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 454/914] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:36.300707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed645cf-22d5-4518-9155-0de1f5527037 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:36.414060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492c8af-5385-44c9-aa1c-492020531d90 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:36.414384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492c8af-5385-44c9-aa1c-492020531d90 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:36.414622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492c8af-5385-44c9-aa1c-492020531d90 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:36.416222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 455/915] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:36.417310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed645cf-22d5-4518-9155-0de1f5527037 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:36.421248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa66b-a4a1-4a40-8050-6f374c94c1d7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:36.421535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa66b-a4a1-4a40-8050-6f374c94c1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:36.421815 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffa66b-a4a1-4a40-8050-6f374c94c1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:36.440368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e2ffa66b-a4a1-4a40-8050-6f374c94c1d7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:36.441573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e2ffa66b-a4a1-4a40-8050-6f374c94c1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:36.457304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:36.457944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:36.458218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:36.458629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:36.459217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:36.460872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:36.472913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:36.473898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 05 16:49:36.474719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:36.479025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 461/916] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:36.481392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:36.520431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 456/917] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:36.547241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:36.592270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2226b8-a152-4839-b7b5-a2aa01871dc7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:36.627007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:36.627788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:36.628062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:36.628360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:36.628987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:36.629284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:36.630637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:36.640206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 462/918] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:36.645258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffa66b-a4a1-4a40-8050-6f374c94c1d7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:36.747017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffa66b-a4a1-4a40-8050-6f374c94c1d7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:36.772535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:36.782940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:36.783879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:36.784615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:36.785284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:36.785983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:36.786538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:36.786618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:36.787750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:36.796252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 457/919] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Sep 5 16:49:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:36.801106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:36.893897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44398f-ee2e-40ae-8a96-7b0856d2e776 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:36.918415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:36.930207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac318a-53b8-4154-802e-e375ed6ef836 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:36.931052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:36.931802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:36.932211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:36.932590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:36.933297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:36.934846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:36.944337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:36.945601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:36.946663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:36.953538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 463/920] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:36.954643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:36.971664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5444a6c-1d27-45e7-b46f-65cfd15d3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:36.972848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5444a6c-1d27-45e7-b46f-65cfd15d3f2e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:36.973126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5444a6c-1d27-45e7-b46f-65cfd15d3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:36.973530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5444a6c-1d27-45e7-b46f-65cfd15d3f2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:36.974379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5444a6c-1d27-45e7-b46f-65cfd15d3f2e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:36.976440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5444a6c-1d27-45e7-b46f-65cfd15d3f2e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:37.008255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 458/921] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:37.072914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cfa7a-209f-4427-8c23-68379afefae7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:37.108888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9526-169d-464f-bfc2-a8d53a9d48ca None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.109264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9526-169d-464f-bfc2-a8d53a9d48ca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.109726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9526-169d-464f-bfc2-a8d53a9d48ca None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.111602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 459/922] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:37.117497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de47e-bcee-4d47-a700-707d5080712b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.117891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de47e-bcee-4d47-a700-707d5080712b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.118218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de47e-bcee-4d47-a700-707d5080712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.129178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:37.129793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.130066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.130332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.130961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:37.131259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:37.133661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:37.137444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3b4de47e-bcee-4d47-a700-707d5080712b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:37.138669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3b4de47e-bcee-4d47-a700-707d5080712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:37.139578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 464/923] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:37.147040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:37.187136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:37.215008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:37.228749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 460/924] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:37.277609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09090d9-d0ef-415a-a59f-22b37f7ca415 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:37.317192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:37.318077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.318583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.319008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.319793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:37.320299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:37.321834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:37.327667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 465/925] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:37.334033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4de47e-bcee-4d47-a700-707d5080712b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:37.440638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4de47e-bcee-4d47-a700-707d5080712b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:37.476528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:37.477482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.477915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.478337 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.479192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] RBAC: Adding query filter params (interface=public, service_id=e4872d7c41fe45c8aeb27088a16ce970, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:37.479653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:37.481561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:37.493605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 461/926] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:49:37 2022] GET /identity/v3/endpoints?service_id=e4872d7c41fe45c8aeb27088a16ce970&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:37.499383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:37.522432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:37.588527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:37.608663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb272545-698e-4ae1-a9ac-5b63f7761a4a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:37.622953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:37.631068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b778a3a-6b80-4ffb-9376-407155b60656 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:37.645666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:37.646615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.647088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.647568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.648409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:37.649978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:37.656690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 466/927] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:37 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:49:37.676273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9424b8c-d33e-4a3e-a3f3-ba688a58d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:37.677693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9424b8c-d33e-4a3e-a3f3-ba688a58d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.678179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9424b8c-d33e-4a3e-a3f3-ba688a58d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.678652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9424b8c-d33e-4a3e-a3f3-ba688a58d7e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.679455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9424b8c-d33e-4a3e-a3f3-ba688a58d7e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:37.681214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9424b8c-d33e-4a3e-a3f3-ba688a58d7e7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d9424b8c-d33e-4a3e-a3f3-ba688a58d7e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:37.692205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:37.693288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 05 16:49:37.694242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:37.702150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 462/928] 10.222.0.23 () {56 vars in 1132 bytes} [Mon Sep 5 16:49:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:37.704039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:37.816269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189db287-d1df-4908-ba49-91ccc0a1422b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:37.819604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5772452-e535-4f0f-96b5-fbd1fb53af6a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.819604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5772452-e535-4f0f-96b5-fbd1fb53af6a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.819604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5772452-e535-4f0f-96b5-fbd1fb53af6a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.821218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 463/929] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:37.826149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310c38e-c3a2-479c-9fd4-70de6ee652d2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.826429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310c38e-c3a2-479c-9fd4-70de6ee652d2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.826703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310c38e-c3a2-479c-9fd4-70de6ee652d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.851933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:37.852570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:37.852844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:37.853110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:37.853728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:37.854027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:37.855684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:37.860706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 467/930] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:37.861191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d310c38e-c3a2-479c-9fd4-70de6ee652d2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:37.862733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d310c38e-c3a2-479c-9fd4-70de6ee652d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:37.868267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:37.916396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:37.937327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 464/931] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:37.982476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a7dbd-d5dc-4cbb-9ea2-4db3e112da5e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:38.018494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:38.019623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:38.020101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:38.020652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:38.021469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:38.022025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:38.023604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:38.034055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 468/932] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:38.035787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310c38e-c3a2-479c-9fd4-70de6ee652d2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:38.067365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:38.139832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310c38e-c3a2-479c-9fd4-70de6ee652d2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:38.180022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37991707-d208-468f-9504-904032750245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:38.180678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37991707-d208-468f-9504-904032750245 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:38.181027 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37991707-d208-468f-9504-904032750245 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:38.181309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37991707-d208-468f-9504-904032750245 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:38.181991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37991707-d208-468f-9504-904032750245 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:38.183378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37991707-d208-468f-9504-904032750245 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:38.209700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 465/933] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:49:38.233561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:38.345917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9d96c-a762-4a46-8706-618456aa6747 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:38.348611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:38.392185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:38.392905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:38.393371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:38.393575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:38.405811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:38.411216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:38.412129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:38.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 469/934] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Sep 5 16:49:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:38.418893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37991707-d208-468f-9504-904032750245 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:38.512753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:38.527042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37991707-d208-468f-9504-904032750245 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:38.562059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:38.562675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:38.562956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:38.563231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:38.563892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:38.564220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:38.565651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:38.574439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 466/935] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Sep 5 16:49:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:38.580051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:38.689925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c368b50-2212-40ec-bd9e-37ab6bea54c9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:38.725966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:38.726997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:38.727477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:38.727952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:38.742053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:38.743521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:38.758507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:38.759520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:38.760917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:38.760917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:38.760917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:38.760917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 470/936] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:38.765529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:38.868682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5670df-9163-447b-80f4-664739ac3f13 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:38.904457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:38.905080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:38.905360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:38.905633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:38.906285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:38.906589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:38.907974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:38.920691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 467/937] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:38.929514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:38.945921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:39.041054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1b5b-fbd2-4ac5-8291-a819654d12b4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.076040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316900-6669-4213-aa6e-1264917f4c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:39.076713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316900-6669-4213-aa6e-1264917f4c5f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.076992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316900-6669-4213-aa6e-1264917f4c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.077265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316900-6669-4213-aa6e-1264917f4c5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.086242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28316900-6669-4213-aa6e-1264917f4c5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:39.087679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28316900-6669-4213-aa6e-1264917f4c5f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-28316900-6669-4213-aa6e-1264917f4c5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:39.099222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:39.100185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.102289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 471/938] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:49:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:39.109257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:39.124526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:39.171482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252dff-ce08-4bde-9668-4a42d0b2ad70 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.171832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252dff-ce08-4bde-9668-4a42d0b2ad70 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.172022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252dff-ce08-4bde-9668-4a42d0b2ad70 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.173641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 472/939] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:39.181175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b997741-9cee-48cc-bb0e-c3677b43f202 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.181661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b997741-9cee-48cc-bb0e-c3677b43f202 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.181933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b997741-9cee-48cc-bb0e-c3677b43f202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.200931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7b997741-9cee-48cc-bb0e-c3677b43f202 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:39.202106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b997741-9cee-48cc-bb0e-c3677b43f202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:39.213150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5585ee8-1c47-4909-a4b0-1e8649f5f499 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.257225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:39.258753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.259347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.259818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.261041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:39.262064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:39.264278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:39.286073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 473/940] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:39.288190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 468/941] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Sep 5 16:49:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:39.293443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b997741-9cee-48cc-bb0e-c3677b43f202 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:39.296033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:39.402004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b997741-9cee-48cc-bb0e-c3677b43f202 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.437972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:39.438612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.438888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.439158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.450052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:39.451624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:39.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:39.467933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:39.468834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:39.468834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:39.468834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.468834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 474/942] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:39.475657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:39.548938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:39.567104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:39.584695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86823dc0-d4de-4896-bb27-1d8345b91426 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.619609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcc44e-d677-49f4-ab4c-f247f808ac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:39.620616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcc44e-d677-49f4-ab4c-f247f808ac0a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.621113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcc44e-d677-49f4-ab4c-f247f808ac0a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.621581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcc44e-d677-49f4-ab4c-f247f808ac0a None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.622459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcc44e-d677-49f4-ab4c-f247f808ac0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:39.622976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcc44e-d677-49f4-ab4c-f247f808ac0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:39.624604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcc44e-d677-49f4-ab4c-f247f808ac0a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:39.637065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:39.640312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 469/943] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:39.643353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815deb4a-4ee0-4c69-9d42-ce0341300464 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.644068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815deb4a-4ee0-4c69-9d42-ce0341300464 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.644924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815deb4a-4ee0-4c69-9d42-ce0341300464 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.650358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 470/944] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:39.650358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815deb4a-4ee0-4c69-9d42-ce0341300464 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:39.683906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e6beb-07aa-44c0-a07b-86ef1bc25268 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.718103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:39.718620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.718908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.719184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.719840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:39.721480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:39.724117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING keystone.server.flask.common [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 05 16:49:39.742917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.sql.core [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 16:49:39.742917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 16:49:39.742917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [parameters: {'id': '6080608fbdfc4c049d521614cf1a5217', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 16:49:39.742917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81522) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2241ba32-2665-4a54-acee-db55e2f609ca None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6080608fbdfc4c049d521614cf1a5217.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6080608fbdfc4c049d521614cf1a5217. Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application conn.query(q) Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 16:49:39.749792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result.read() Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 05 16:49:39.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application next(self.gen) Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application yield resource Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application next(self.gen) Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise self.value Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application session.commit() Sep 05 16:49:39.751620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.session.flush() Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application compat.raise_( Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 16:49:39.753257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 16:49:39.754156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application conn.query(q) Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application result.read() Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 16:49:39.754938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [parameters: {'id': '6080608fbdfc4c049d521614cf1a5217', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:39.755830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:39.758948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6080608fbdfc4c049d521614cf1a5217. Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 475/945] 10.222.0.23 () {60 vars in 1140 bytes} [Mon Sep 5 16:49:39 2022] POST /identity/v3/projects => generated 262 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45357c-c3f7-479f-a965-37236df45d52 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.761451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45357c-c3f7-479f-a965-37236df45d52 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.764043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45357c-c3f7-479f-a965-37236df45d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.764655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815deb4a-4ee0-4c69-9d42-ce0341300464 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.773744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5a45357c-c3f7-479f-a965-37236df45d52 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:39.775314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5a45357c-c3f7-479f-a965-37236df45d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:39.792562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:39.802081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:39.802664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.803133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.803422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.810377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:39.811832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:39.823492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:39.824523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:39.825410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 471/946] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:39.825921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:39.854217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 476/947] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:39.855651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45357c-c3f7-479f-a965-37236df45d52 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:39.901828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:39.943305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30af43-2d58-4239-951a-f7e68ae88880 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.965163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45357c-c3f7-479f-a965-37236df45d52 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:39.979128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:39.980579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:39.981126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:39.981597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:39.989382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:39.991301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.002523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:40.003146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.003423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.003691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.004453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:40.004760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:40.005209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:40.006097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:40.006986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:40.007442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.009754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 472/948] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:40.020212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.027372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 477/949] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:40.029779 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.099304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:40.177455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba238925-2d9a-4d33-92de-5e257bec2bd7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:40.189278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:40.193060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92450a-1651-451c-83f9-00935ca73a0f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:40.214914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:40.215549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.215866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.222229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.222229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:40.222229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:40.222229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.236653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b4335-8d5f-49dd-b59b-cfe0626aae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:40.237371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b4335-8d5f-49dd-b59b-cfe0626aae53 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.248652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b4335-8d5f-49dd-b59b-cfe0626aae53 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.248652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b4335-8d5f-49dd-b59b-cfe0626aae53 None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2f2310a3da094b959270d797bd2f1553/roles/59ee14375b7b400983f97480ca2fe124` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.249248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 473/950] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:40.249248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.250283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b4335-8d5f-49dd-b59b-cfe0626aae53 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=2f2310a3da094b959270d797bd2f1553, role_id=59ee14375b7b400983f97480ca2fe124)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:40.250283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b4335-8d5f-49dd-b59b-cfe0626aae53 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.270151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 478/951] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:40 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2f2310a3da094b959270d797bd2f1553/roles/59ee14375b7b400983f97480ca2fe124 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:49:40.326282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cc574-508e-44f9-a378-73ceb4762387 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.326546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cc574-508e-44f9-a378-73ceb4762387 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.326837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cc574-508e-44f9-a378-73ceb4762387 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.328733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 479/952] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:40.335938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab1cd5-5aba-4789-a920-65993ebc2a58 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.336253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab1cd5-5aba-4789-a920-65993ebc2a58 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.336538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab1cd5-5aba-4789-a920-65993ebc2a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.363195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9aab1cd5-5aba-4789-a920-65993ebc2a58 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:40.364141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9aab1cd5-5aba-4789-a920-65993ebc2a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:40.417394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cbcd0-3c06-4b00-a92f-21f8d861d6da None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:40.433373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:40.466932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 480/953] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:40.472476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:40.473360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.473809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.474224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.474966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:40.476508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:40.486220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:40.487570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:40.488785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:40.490583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 474/954] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:40.496750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab1cd5-5aba-4789-a920-65993ebc2a58 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.591034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:40.612323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab1cd5-5aba-4789-a920-65993ebc2a58 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:40.647588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:40.648512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.648965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.649383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.650164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:40.650621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:40.652137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.657934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 481/955] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:40.664122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.743528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.775829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3beeee-5162-4b95-b573-db3d997edd8b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:40.817868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:40.818487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.818765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.819037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.819669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:40.819973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:40.821693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.833201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 475/956] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:40.837083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.870308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeba85-5d88-4ff7-bf98-06decdbb407f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:40.883428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:40.916138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:40.916948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:40.917244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:40.917614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:40.918392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:40.919887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:40.931432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:40.932800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:40.933871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:40.939975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 482/957] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:40.941897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:40.962828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:40.974878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45316b6-a0f6-44b4-b524-2f683f29c717 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:41.021505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:41.022519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:41.023034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:41.023504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:41.024417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:41.026414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:41.053533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb65df-a2cc-4646-8692-f4e4dbb5cf7c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:41.060967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 476/958] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:41.108567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b03adc-ced0-4c22-a1e3-65de7b9109c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:41.109189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03adc-ced0-4c22-a1e3-65de7b9109c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:41.109558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03adc-ced0-4c22-a1e3-65de7b9109c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:41.109835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03adc-ced0-4c22-a1e3-65de7b9109c5 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:41.110471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b03adc-ced0-4c22-a1e3-65de7b9109c5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:41.110771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b03adc-ced0-4c22-a1e3-65de7b9109c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:41.113116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b03adc-ced0-4c22-a1e3-65de7b9109c5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:41.119355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 483/959] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:41.125900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:41.233228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:41.243505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dfa5e-632b-402d-9a21-aacb7172c2d7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:41.280353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cdaba-9789-4fda-8955-cf98090d4f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:41.281016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cdaba-9789-4fda-8955-cf98090d4f92 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:41.281299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cdaba-9789-4fda-8955-cf98090d4f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:41.281569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cdaba-9789-4fda-8955-cf98090d4f92 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:41.282196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cdaba-9789-4fda-8955-cf98090d4f92 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:41.282498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cdaba-9789-4fda-8955-cf98090d4f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:41.284520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43cdaba-9789-4fda-8955-cf98090d4f92 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:41.289634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 477/960] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:42.254866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729fbd2-770f-4ca2-b291-7fdcc0d4e5c5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:42.255902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729fbd2-770f-4ca2-b291-7fdcc0d4e5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:42.256426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729fbd2-770f-4ca2-b291-7fdcc0d4e5c5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:42.258268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 484/961] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:42.263684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323021e7-421f-4887-a59b-4ff597c7a9fa None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:42.264543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323021e7-421f-4887-a59b-4ff597c7a9fa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:42.265020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323021e7-421f-4887-a59b-4ff597c7a9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:42.285039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-323021e7-421f-4887-a59b-4ff597c7a9fa None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:42.286639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-323021e7-421f-4887-a59b-4ff597c7a9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:42.359124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 478/962] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:42.386532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd6989-13cc-4246-ac55-7cb8032317c0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:42.386814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd6989-13cc-4246-ac55-7cb8032317c0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:42.387083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd6989-13cc-4246-ac55-7cb8032317c0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:42.388721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 485/963] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:42.393884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dab7f6-6971-4a77-aa09-fc34a27bdf15 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:42.394172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dab7f6-6971-4a77-aa09-fc34a27bdf15 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:42.394451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dab7f6-6971-4a77-aa09-fc34a27bdf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:42.412988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f6dab7f6-6971-4a77-aa09-fc34a27bdf15 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:42.414164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f6dab7f6-6971-4a77-aa09-fc34a27bdf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:42.481841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 479/964] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:42.640831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd6989-13cc-4246-ac55-7cb8032317c0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:42.726574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:42.750270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dab7f6-6971-4a77-aa09-fc34a27bdf15 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:42.751739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd6989-13cc-4246-ac55-7cb8032317c0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:42.768458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:42.790081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:42.791115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:42.791595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:42.792074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:42.803136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:42.808344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:42.809282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:42.810149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:42.812879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 486/965] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Sep 5 16:49:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:42.818908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:42.852060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dab7f6-6971-4a77-aa09-fc34a27bdf15 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:42.887622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:42.888271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:42.888548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:42.888819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:42.889472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:42.890972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:42.893575 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING keystone.server.flask.common [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 05 16:49:42.906923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 16:49:42.906923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 16:49:42.906923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [parameters: {'user_id': 'df8ef4e102a04d95a2d811a87593c6de', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 16:49:42.906923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Sep 05 16:49:42.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 16:49:42.930288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 05 16:49:42.931291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 05 16:49:42.932251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 05 16:49:42.933845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 05 16:49:42.935898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'user_id': 'df8ef4e102a04d95a2d811a87593c6de', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:42.938096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 05 16:49:42.940134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 05 16:49:42.942645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 05 16:49:42.942645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 05 16:49:42.942645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 05 16:49:42.942645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 05 16:49:42.942645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 05 16:49:42.942645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:42.942645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 480/966] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Sep 5 16:49:42 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:42.943673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf7d33-cbe5-4e98-889c-2ad8e511733b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:42.943826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:42.980708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:42.981338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:42.981669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:42.981942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:42.982896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:42.983203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:42.984676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:42.998190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 487/967] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Sep 5 16:49:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:42.998190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.001825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.035899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.058984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db56614-d62d-44b1-821e-b4da1192277a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.097421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4d4d4-5873-4582-80f6-6d6996c1c897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.098029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d4d4-5873-4582-80f6-6d6996c1c897 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.098308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d4d4-5873-4582-80f6-6d6996c1c897 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.098582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4d4d4-5873-4582-80f6-6d6996c1c897 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.106704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4d4d4-5873-4582-80f6-6d6996c1c897 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.108374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4d4d4-5873-4582-80f6-6d6996c1c897 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.115525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3f207-4bc2-4282-8146-5c6e48299a55 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-82e4d4d4-5873-4582-80f6-6d6996c1c897 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:43.119787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:43.120723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 05 16:49:43.121680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:43.125779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 481/968] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:43.127119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e6546-2567-4c40-98c8-20563de3e666 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.127468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e6546-2567-4c40-98c8-20563de3e666 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.127828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e6546-2567-4c40-98c8-20563de3e666 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.132039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 482/969] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:43.132626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e6546-2567-4c40-98c8-20563de3e666 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.158987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51222a-cbe2-4add-b7bc-35276448193a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.159544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51222a-cbe2-4add-b7bc-35276448193a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.159832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51222a-cbe2-4add-b7bc-35276448193a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.160085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51222a-cbe2-4add-b7bc-35276448193a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.170958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51222a-cbe2-4add-b7bc-35276448193a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.172443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51222a-cbe2-4add-b7bc-35276448193a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3c51222a-cbe2-4add-b7bc-35276448193a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:43.189094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:43.190295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:43.191420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:43.191420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:43.191420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:43.196842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 488/970] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:43.196842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b101de9-b615-4ed9-8703-5aa62a173715 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.196842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b101de9-b615-4ed9-8703-5aa62a173715 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.196842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b101de9-b615-4ed9-8703-5aa62a173715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.213081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2b101de9-b615-4ed9-8703-5aa62a173715 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:43.214255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2b101de9-b615-4ed9-8703-5aa62a173715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:43.242298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e6546-2567-4c40-98c8-20563de3e666 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.261576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.278547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1549bc-962f-4829-8a0f-4ada49f4dd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.279163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1549bc-962f-4829-8a0f-4ada49f4dd18 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.279458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1549bc-962f-4829-8a0f-4ada49f4dd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.279709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1549bc-962f-4829-8a0f-4ada49f4dd18 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.280423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1549bc-962f-4829-8a0f-4ada49f4dd18 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:43.280723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1549bc-962f-4829-8a0f-4ada49f4dd18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.282149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1549bc-962f-4829-8a0f-4ada49f4dd18 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.283853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 489/971] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:43.284714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b101de9-b615-4ed9-8703-5aa62a173715 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.300921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.332188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 483/972] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:43.332188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c2d82-e640-4aec-8802-1bbe666c155f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.332188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c2d82-e640-4aec-8802-1bbe666c155f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.332188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c2d82-e640-4aec-8802-1bbe666c155f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.332188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 484/973] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:43.332188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8db02-0143-4ecb-890c-ae41e2f730cb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.332188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8db02-0143-4ecb-890c-ae41e2f730cb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.332800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8db02-0143-4ecb-890c-ae41e2f730cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.350893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-46a8db02-0143-4ecb-890c-ae41e2f730cb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:43.352542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-46a8db02-0143-4ecb-890c-ae41e2f730cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:43.404923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b101de9-b615-4ed9-8703-5aa62a173715 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.451529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.452266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.452867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.453150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.454188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:43.454501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.456167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.457595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 485/974] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:43.476217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 490/975] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:43.485651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8db02-0143-4ecb-890c-ae41e2f730cb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.553516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.608484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8db02-0143-4ecb-890c-ae41e2f730cb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.608709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.644829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.653297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.653939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.654218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.654485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.664659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.666365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.667172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eec7f-12ac-42a1-a73e-7597df6b085b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:43.679592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:43.680579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:43.682022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 486/976] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:43.686425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.703817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.704388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.704665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.704935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.705587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.707115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:43.715957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:43.716968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:43.717821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 491/977] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:43.723826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.790510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e445c4-a5eb-4b52-87f3-302401edaf7f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.824015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248dc05a-3f55-47ff-8660-2f5539b84599 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.824276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.824886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.825165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.825550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.826271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:43.826577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.828046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.831453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.845550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 487/978] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:43.846380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.863887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.864551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.864824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.865093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.865712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:43.866009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.867370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:43.872140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 492/979] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:43.872935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:43.899282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:43.953206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afeb8f3-8ae4-4fca-8183-1841d166902a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.973573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f7011-6cdc-4669-8761-1d6dff4b1032 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:43.987531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:43.988135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:43.988442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:43.988715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:43.989348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:43.990714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.005635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 488/980] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:43 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:44.006442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:44.015115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.015762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.016035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.016372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.055020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:44.093100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.094572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.132978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb246f-4c81-4e56-83a1-97093dbfab10 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:44.136792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:44.176431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.176822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.177098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.177366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.178553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:44.178553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.179651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.191107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 489/981] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:44.192862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:44.210257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 493/982] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:43 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:44.215612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:44.286591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:44.316902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223e914-7db0-4b77-ba75-1010b45b250c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:44.322176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98a7be-1bc5-4c26-9c84-f71c41bd29ab None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:44.356637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.357637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.357911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.358364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.358540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.358821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.359090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.359417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.360357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] RBAC: Adding query filter params (interface=admin, service_id=e4872d7c41fe45c8aeb27088a16ce970, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:44.360993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.362752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.373243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 490/983] 10.222.0.23 () {56 vars in 1249 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/endpoints?service_id=e4872d7c41fe45c8aeb27088a16ce970&interface=admin®ion_id=RegionOne => generated 193 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:44.374436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:44.375877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.377514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:44.395036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:44.396034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:44.402321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:44.402321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:44.402321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:44.402321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 494/984] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:44.402564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:44.412549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:44.488460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49512128-b28f-40b4-aa2a-78bf755b47a5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:44.508996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae217167-685c-4092-b13d-66c4236abaa4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:44.524606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.525219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.525490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.525759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.526467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.527082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:44.528877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.536119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 491/985] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:49:44.554375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.555310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.555770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.556271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.557152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:44.557659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.559243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.577757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 495/986] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:44.578997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:44.690562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:44.696449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc0112-cac7-4277-bf00-4da12cd3f1f7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:44.734934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.735919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.736464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.737060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.744094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.745797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:44.757277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:44.758253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:44.759158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:44.759603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 492/987] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:44.765911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:44.870383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84234eaa-d4cd-4bde-9cbf-a62c468e3401 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:44.906767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299d2c7-fb9a-4734-a8ce-a69c02e05d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:44.907371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299d2c7-fb9a-4734-a8ce-a69c02e05d14 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:44.907649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299d2c7-fb9a-4734-a8ce-a69c02e05d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:44.907919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299d2c7-fb9a-4734-a8ce-a69c02e05d14 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:44.908709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299d2c7-fb9a-4734-a8ce-a69c02e05d14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:44.909059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299d2c7-fb9a-4734-a8ce-a69c02e05d14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:44.910499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299d2c7-fb9a-4734-a8ce-a69c02e05d14 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:44.923296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 496/988] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:44.929586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:45.039589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c957a-90bc-4909-a674-2e47d85d8870 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:45.079896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe654c4-94c0-4cb5-ac47-0f0f38aad0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:45.080756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe654c4-94c0-4cb5-ac47-0f0f38aad0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:45.081204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe654c4-94c0-4cb5-ac47-0f0f38aad0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:45.081555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe654c4-94c0-4cb5-ac47-0f0f38aad0c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:45.082303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe654c4-94c0-4cb5-ac47-0f0f38aad0c0 None admin] RBAC: Adding query filter params (role.id=59ee14375b7b400983f97480ca2fe124, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2f2310a3da094b959270d797bd2f1553) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:45.082612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe654c4-94c0-4cb5-ac47-0f0f38aad0c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:45.084085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe654c4-94c0-4cb5-ac47-0f0f38aad0c0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:45.091088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 493/989] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:44 2022] GET /identity/v3/role_assignments?user.id=2f2310a3da094b959270d797bd2f1553&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=59ee14375b7b400983f97480ca2fe124 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:45.902944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f79496-096a-4843-8965-1d4d818a02b8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:45.903522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f79496-096a-4843-8965-1d4d818a02b8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:45.903522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f79496-096a-4843-8965-1d4d818a02b8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:45.905064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 497/990] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:45.910147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ee24-024d-4a80-bc06-d64fc2a1d6ac None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:45.910586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ee24-024d-4a80-bc06-d64fc2a1d6ac None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:45.910684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8ee24-024d-4a80-bc06-d64fc2a1d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:45.929442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-37d8ee24-024d-4a80-bc06-d64fc2a1d6ac None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:45.930698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-37d8ee24-024d-4a80-bc06-d64fc2a1d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:45.946763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:46.005234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 494/991] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:46.269028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f79496-096a-4843-8965-1d4d818a02b8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:46.322654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:46.395718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f79496-096a-4843-8965-1d4d818a02b8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:46.433140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:46.433772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:46.434020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:46.434319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:46.434898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:46.436383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:46.437347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536c44-c32d-40d4-a1a8-c9e2bc1a153c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:46.437620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536c44-c32d-40d4-a1a8-c9e2bc1a153c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:46.437931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43536c44-c32d-40d4-a1a8-c9e2bc1a153c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:46.439365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 495/992] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:46.444263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4407cb-d357-423f-a9e1-65b39867209b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:46.444551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4407cb-d357-423f-a9e1-65b39867209b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:46.444826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4407cb-d357-423f-a9e1-65b39867209b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:46.446434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:46.447382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:46.448278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 498/993] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:46.453546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:46.465899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4d4407cb-d357-423f-a9e1-65b39867209b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:46.467106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4d4407cb-d357-423f-a9e1-65b39867209b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:46.540442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 496/994] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:46.561256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf8e62-9bc4-42ff-bdd9-b2283ed4ede2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:46.562928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:46.602045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:46.602653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:46.602929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:46.603198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:46.603822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:46.604206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:46.605879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:46.615321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 499/995] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:46.617140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4407cb-d357-423f-a9e1-65b39867209b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:46.706974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:46.727978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4407cb-d357-423f-a9e1-65b39867209b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:46.763014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:46.763655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:46.763935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:46.764245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:46.764877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:46.765181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:46.766633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:46.773007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 497/996] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:46.779035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:46.805096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:46.889736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00024e09-5273-4ae2-92a4-078ffdb1b470 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:46.911965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691c88-ecfa-412b-b56a-6cf6cdf366e3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:46.927045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:46.927658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:46.927939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:46.928237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:46.928939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] RBAC: Adding query filter params (interface=public, service_id=c577625d15c2488aaec69aac82c190cd, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:46.929358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:46.930831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:46.936560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 500/997] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:49:46 2022] GET /identity/v3/endpoints?service_id=c577625d15c2488aaec69aac82c190cd&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:46.942365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:46.951931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:46.952612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:46.952883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:46.953366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:46.953957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:46.955518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:46.964353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:46.965507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:46.966331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 498/998] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Sep 5 16:49:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:46.971920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:46.978074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:46.983403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:47.050013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536bfdbd-cf09-4247-9360-42ed6456038c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:47.075488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a54af-472e-44be-8864-3d90e07df0c5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:47.084141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61a381-8a68-4c23-baf9-3912741713b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:47.084776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a381-8a68-4c23-baf9-3912741713b3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.085052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a381-8a68-4c23-baf9-3912741713b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.085323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61a381-8a68-4c23-baf9-3912741713b3 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.085922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61a381-8a68-4c23-baf9-3912741713b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:47.087291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61a381-8a68-4c23-baf9-3912741713b3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:47.093587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 501/999] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:46 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 16:49:47.094797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f8621-3759-4e1a-bd20-39cbaa67e8f4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.095086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f8621-3759-4e1a-bd20-39cbaa67e8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.095355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f8621-3759-4e1a-bd20-39cbaa67e8f4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.096711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 502/1000] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:47.110550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfbb51-3af9-4e2f-8003-9ead2a3f9647 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.110909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfbb51-3af9-4e2f-8003-9ead2a3f9647 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.111373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfbb51-3af9-4e2f-8003-9ead2a3f9647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.115234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:47.115873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.116181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.116457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.117155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:47.117399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:47.118823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:47.124961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 499/1001] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:47.130673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:47.132306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5fdfbb51-3af9-4e2f-8003-9ead2a3f9647 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:47.133531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5fdfbb51-3af9-4e2f-8003-9ead2a3f9647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:47.209726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:47.211729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 503/1002] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:47.243938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baca21-36b9-4f83-8019-11dd28519167 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:47.279874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:47.280523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.280808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.281083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.281715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:47.282019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:47.283418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:47.288767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 500/1003] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Sep 5 16:49:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:47.294391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfbb51-3af9-4e2f-8003-9ead2a3f9647 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:47.319197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:47.401100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfbb51-3af9-4e2f-8003-9ead2a3f9647 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:47.437125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:47.437739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.438019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.438288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.438963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:47.440420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:47.460338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 504/1004] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:47.492454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:47.604495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339194e-7a5d-4f0e-85de-bcd1608cc428 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:47.642360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:47.648008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:47.649012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.649546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.650060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] PATH_INFO: `/v3/services/image` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.650867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:47.652511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:47.662400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:47.663386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:47.664327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:47.666156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 501/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Mon Sep 5 16:49:47 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:47.671949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:47.761212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:47.779946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229454ec-9bf9-41a6-a19b-384175d567fe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:47.815197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82252369-15e4-4461-8132-64744d11df90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:47.816191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82252369-15e4-4461-8132-64744d11df90 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.816670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82252369-15e4-4461-8132-64744d11df90 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.817148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82252369-15e4-4461-8132-64744d11df90 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.817964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82252369-15e4-4461-8132-64744d11df90 None admin] RBAC: Adding query filter params (name=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:47.818462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82252369-15e4-4461-8132-64744d11df90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:47.820088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82252369-15e4-4461-8132-64744d11df90 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:47.825400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 505/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:47.830938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:47.944571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f330a-8fd2-47a9-94ae-6c03863f3f73 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:47.980587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d72f8-f903-4a53-b2c2-30fc42ef0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:47.981183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d72f8-f903-4a53-b2c2-30fc42ef0785 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:47.981463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d72f8-f903-4a53-b2c2-30fc42ef0785 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:47.981731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d72f8-f903-4a53-b2c2-30fc42ef0785 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:47.982350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d72f8-f903-4a53-b2c2-30fc42ef0785 None admin] RBAC: Adding query filter params (type=image) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:47.982647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d72f8-f903-4a53-b2c2-30fc42ef0785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:47.984022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d72f8-f903-4a53-b2c2-30fc42ef0785 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:47.989006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 502/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:49.023781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fa345-119e-4abf-92c6-d5fb406ce40b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.024346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fa345-119e-4abf-92c6-d5fb406ce40b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.024346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6fa345-119e-4abf-92c6-d5fb406ce40b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.025762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 506/1008] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:49.031196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e309f-444c-468d-aa08-1a2702df955a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.031628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e309f-444c-468d-aa08-1a2702df955a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.031704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e309f-444c-468d-aa08-1a2702df955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.052179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c57e309f-444c-468d-aa08-1a2702df955a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:49.053486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c57e309f-444c-468d-aa08-1a2702df955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:49.113603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:49.126805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 503/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:49.351757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c72a6-8538-4a5a-be5d-1626df00c6f4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.352122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c72a6-8538-4a5a-be5d-1626df00c6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.352346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c72a6-8538-4a5a-be5d-1626df00c6f4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.353858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 507/1010] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:49.359387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359f79-6a57-4c6c-8de8-ecf233926314 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.359680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359f79-6a57-4c6c-8de8-ecf233926314 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.359955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26359f79-6a57-4c6c-8de8-ecf233926314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.383939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-26359f79-6a57-4c6c-8de8-ecf233926314 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:49.385216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26359f79-6a57-4c6c-8de8-ecf233926314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:49.393726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c72a6-8538-4a5a-be5d-1626df00c6f4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:49.458858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 504/1011] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:49.504143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c72a6-8538-4a5a-be5d-1626df00c6f4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:49.539391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:49.540019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.540323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.540591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] PATH_INFO: `/v3/services/compute` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.541160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:49.542506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:49.551519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:49.552527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:49.557177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 508/1012] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:49.558865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26359f79-6a57-4c6c-8de8-ecf233926314 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:49.662612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26359f79-6a57-4c6c-8de8-ecf233926314 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:49.683086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:49.699047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:49.699655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.699932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.700227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.700853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:49.701162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:49.702538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:49.708062 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 505/1013] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:49.713770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:49.729481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:49.740037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:49.819232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a21df-5853-4bcf-85d7-a76ac895caed None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:49.845710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69de3b-1569-4a3d-9c21-267bc290c32f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:49.853327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b24ea-b866-40ad-aba1-00fe16813094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:49.853996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b24ea-b866-40ad-aba1-00fe16813094 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.854271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b24ea-b866-40ad-aba1-00fe16813094 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.854539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b24ea-b866-40ad-aba1-00fe16813094 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.855155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b24ea-b866-40ad-aba1-00fe16813094 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:49.855770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b24ea-b866-40ad-aba1-00fe16813094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:49.857153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b24ea-b866-40ad-aba1-00fe16813094 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:49.863212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 509/1014] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Sep 5 16:49:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:49.863365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9e5e-a149-42a6-885d-b9d0c296a128 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.864183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9e5e-a149-42a6-885d-b9d0c296a128 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.864183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa9e5e-a149-42a6-885d-b9d0c296a128 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.867305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 510/1015] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:49.869026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9e5e-a149-42a6-885d-b9d0c296a128 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:49.881190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6054974-11ee-4ded-8e73-b26089b8f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:49.881827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6054974-11ee-4ded-8e73-b26089b8f825 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.882107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6054974-11ee-4ded-8e73-b26089b8f825 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.882375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6054974-11ee-4ded-8e73-b26089b8f825 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.883478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6054974-11ee-4ded-8e73-b26089b8f825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:49.884945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6054974-11ee-4ded-8e73-b26089b8f825 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:49.886524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 506/1016] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:49.887690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdefe1-0173-4eb4-87ea-25245630da12 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:49.887971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdefe1-0173-4eb4-87ea-25245630da12 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:49.888410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdefe1-0173-4eb4-87ea-25245630da12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:49.906603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-eabdefe1-0173-4eb4-87ea-25245630da12 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:49.907792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eabdefe1-0173-4eb4-87ea-25245630da12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:49.945633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:49.954592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:49.969316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa9e5e-a149-42a6-885d-b9d0c296a128 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:49.976551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 507/1017] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:49.977350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdefe1-0173-4eb4-87ea-25245630da12 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:50.010872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:50.011474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:50.011765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:50.012033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:50.012734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:50.014071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:50.034888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 511/1018] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:49:50.087442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdefe1-0173-4eb4-87ea-25245630da12 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:50.133445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4b3e1-dbf7-4bfe-b9be-70ec549b6edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:50.134065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4b3e1-dbf7-4bfe-b9be-70ec549b6edc None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:50.134343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4b3e1-dbf7-4bfe-b9be-70ec549b6edc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:50.134618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4b3e1-dbf7-4bfe-b9be-70ec549b6edc None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:50.135264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4b3e1-dbf7-4bfe-b9be-70ec549b6edc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:50.136762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4b3e1-dbf7-4bfe-b9be-70ec549b6edc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:50.172190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 508/1019] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:49 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:50.247525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:50.359427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fdfa8-2fd2-43fb-9263-2e8640ad1c20 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:50.399762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:50.405545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc6010-db02-4ad9-9cba-e4fa6a001db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:50.406524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc6010-db02-4ad9-9cba-e4fa6a001db5 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:50.406645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc6010-db02-4ad9-9cba-e4fa6a001db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:50.406872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc6010-db02-4ad9-9cba-e4fa6a001db5 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:50.407920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc6010-db02-4ad9-9cba-e4fa6a001db5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:50.410148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc6010-db02-4ad9-9cba-e4fa6a001db5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:50.427119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 512/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:49:50 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:52.029357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56c3f7-3e45-4fa9-be95-c7b998863c2e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.029894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56c3f7-3e45-4fa9-be95-c7b998863c2e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.029894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56c3f7-3e45-4fa9-be95-c7b998863c2e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.031273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 509/1021] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:52.036722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58bb3a-e148-4fc2-8cdc-c30a436f485c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.037143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58bb3a-e148-4fc2-8cdc-c30a436f485c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.037217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58bb3a-e148-4fc2-8cdc-c30a436f485c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.055979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9f58bb3a-e148-4fc2-8cdc-c30a436f485c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:52.057183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9f58bb3a-e148-4fc2-8cdc-c30a436f485c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:52.126401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e2293-6bc1-48a5-bd3c-d76ff957c32a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.126677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e2293-6bc1-48a5-bd3c-d76ff957c32a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.126958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e2293-6bc1-48a5-bd3c-d76ff957c32a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.128490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 510/1022] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:52.133495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bb5cc-205f-4563-9dcf-3d048e878edb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.133780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bb5cc-205f-4563-9dcf-3d048e878edb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.134052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bb5cc-205f-4563-9dcf-3d048e878edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.138978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 513/1023] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:52.152703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b18bb5cc-205f-4563-9dcf-3d048e878edb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:52.153894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b18bb5cc-205f-4563-9dcf-3d048e878edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:52.178318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:52.231322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 511/1024] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:52.313157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c66f-e056-41bd-a2fe-2522ba10afc8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.313341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c66f-e056-41bd-a2fe-2522ba10afc8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.313592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03c66f-e056-41bd-a2fe-2522ba10afc8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.315186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 514/1025] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:52.320422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4293f-a5b4-4165-af72-f06261a0beec None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.320693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4293f-a5b4-4165-af72-f06261a0beec None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.320993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe4293f-a5b4-4165-af72-f06261a0beec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.339748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8fe4293f-a5b4-4165-af72-f06261a0beec None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:52.340961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8fe4293f-a5b4-4165-af72-f06261a0beec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:52.408580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03c66f-e056-41bd-a2fe-2522ba10afc8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:52.421685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 512/1026] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:52.501901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4293f-a5b4-4165-af72-f06261a0beec None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:52.517255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03c66f-e056-41bd-a2fe-2522ba10afc8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:52.537875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:52.556844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:52.557468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.557740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.558014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.568996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:52.574024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:52.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:52.577094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 515/1027] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Sep 5 16:49:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:52.580910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:52.597132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:52.609126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe4293f-a5b4-4165-af72-f06261a0beec None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:52.645556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:52.646615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.647081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.647633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.648753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:52.650196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:52.656586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 513/1028] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:52.657671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:52.681774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f8456-8625-4250-9302-c84b18efe972 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:52.717101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:52.717707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.717986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.718255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.718911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:52.719212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:52.720624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:52.734342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 516/1029] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Sep 5 16:49:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:52.734342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:52.761933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f540ef8-7fa1-4737-9f11-5888a3072c7f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:52.772428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:52.797662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:52.798277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.798555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.798827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.799637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:52.801103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:52.823871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 514/1030] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:49:52 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:52.824732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:52.836264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2964b4-8da4-4b8f-9f21-55b3be80296e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:52.848527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:52.875220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:52.875879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:52.876180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:52.876464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] PATH_INFO: `/v3/services/image` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:52.877106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:52.878708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:52.888397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:52.889492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:52.890325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 517/1031] 10.222.0.23 () {56 vars in 1105 bytes} [Mon Sep 5 16:49:52 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:52.896629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:52.959355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf28117-d314-4c05-93c2-59af8f50b0f8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:52.999240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:52.999884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.000189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.000462 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.012055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.013637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.021056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb366b1d-b73b-43f6-a844-569dff6f7fce None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.025798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:53.030808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:53.032187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:53.033085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:53.033085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:53.033085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:53.033085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 515/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:53.039646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:53.066000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.066540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.066759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.066916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.067590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] RBAC: Adding query filter params (name=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:53.067895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.069290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.075232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 518/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:53.080825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:53.117748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:53.161847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720b6ce-febc-41e1-aded-d091434b59e0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.189110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fe122-8f40-4000-b5a4-b850eba8b07e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.200168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.200815 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.201114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.201692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.202343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:53.202640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.204024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.217545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 516/1034] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:53.223391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:53.230751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.231358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.231623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.231890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.232547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] RBAC: Adding query filter params (type=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:53.232847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.234205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.239412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 519/1035] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:53.245045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:53.279091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:53.333363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0189-5dd7-4b61-be3d-22a982a6b0f4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.359626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e2922-ac33-4990-9e02-f7e2e2cd8225 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.360762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:53.374007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.374660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.374939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.375210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.383450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.384950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:53.397554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:53.398555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:53.399510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 517/1036] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:53.402404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.403076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.403496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.403799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.404536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] RBAC: Adding query filter params (interface=public, service_id=781c6f8e0d554a6db8447181d0038b26, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:53.404845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.405044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:53.406192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.416234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 520/1037] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/endpoints?service_id=781c6f8e0d554a6db8447181d0038b26&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:53.417069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:53.511055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6392c-ec20-43db-a9a3-ecfd0a365980 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.518928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:53.522154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedab1c1-025f-4d35-ac67-9ee598ed837a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.550771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.551384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.551660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.551926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.552667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:53.552970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.554392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.564273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.565041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.565323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.565754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.566348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.567701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.575970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 518/1038] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:53.576167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 521/1039] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:49:53.577104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:53.630415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:53.692934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab242c-891c-4bd1-9331-d9ec3ee2305b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:53.733141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:53.733765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:53.734052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:53.734333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:53.804679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:53.806148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:53.851408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:53.908865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 519/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:53.914497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:54.022266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d3d2a-ca4d-4fac-8ad6-42e8955270bc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:54.056530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:54.057182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:54.057461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:54.057732 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:54.069318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:54.070765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:54.085478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:54.086558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:54.092613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:54.092613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:54.092613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:54.092613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 522/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:54.094086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:54.200181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de037c-a45e-4bba-b603-e339efec6b65 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:54.235123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:54.235742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:54.236022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:54.236378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:54.237056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:54.237366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:54.238745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:54.252687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 520/1042] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:54.255821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:54.271944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:54.360837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adfcdb-b37a-401e-96ae-cb7f74da5f1f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:54.402717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:54.403748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:54.404275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:54.404771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:54.412106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:54.413845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:54.426136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:54.427139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:54.428030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:54.433678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 523/1043] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:54.434923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:54.440388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:54.545112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b289-d2c3-44ea-beee-f1a54850c9fc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:54.580187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e447023-ed5d-4334-8bb8-6ef41a871566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:54.581199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447023-ed5d-4334-8bb8-6ef41a871566 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:54.581679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447023-ed5d-4334-8bb8-6ef41a871566 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:54.582131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e447023-ed5d-4334-8bb8-6ef41a871566 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:54.583008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447023-ed5d-4334-8bb8-6ef41a871566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:54.583500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447023-ed5d-4334-8bb8-6ef41a871566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:54.585149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e447023-ed5d-4334-8bb8-6ef41a871566 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:54.598121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 521/1044] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:54.604103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:54.682147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:54.713697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4459c-cfa4-4c1d-badd-5c4314b4f99c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:54.748616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c16816-925e-4e76-b6a8-50c0837dd1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:54.749218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16816-925e-4e76-b6a8-50c0837dd1eb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:54.749490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16816-925e-4e76-b6a8-50c0837dd1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:54.749757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c16816-925e-4e76-b6a8-50c0837dd1eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:54.750475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c16816-925e-4e76-b6a8-50c0837dd1eb None admin] RBAC: Adding query filter params (role.id=59ee14375b7b400983f97480ca2fe124, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=c30e20ce4d63452095aedb60249f0cce) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:54.750809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c16816-925e-4e76-b6a8-50c0837dd1eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:54.752374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c16816-925e-4e76-b6a8-50c0837dd1eb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:54.757970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 524/1045] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:54 2022] GET /identity/v3/role_assignments?user.id=c30e20ce4d63452095aedb60249f0cce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=59ee14375b7b400983f97480ca2fe124 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:54.857854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2d233-1871-4aae-9338-d3c69fa5583b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:54.858173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2d233-1871-4aae-9338-d3c69fa5583b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:54.858570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2d233-1871-4aae-9338-d3c69fa5583b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:54.860056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 522/1046] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:54.866134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afbae1-8f20-412d-8074-15c1b222a8fd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:54.866465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afbae1-8f20-412d-8074-15c1b222a8fd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:54.866688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afbae1-8f20-412d-8074-15c1b222a8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:54.886543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-98afbae1-8f20-412d-8074-15c1b222a8fd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:54.887755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-98afbae1-8f20-412d-8074-15c1b222a8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:54.959823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 525/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:55.224873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2d233-1871-4aae-9338-d3c69fa5583b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:55.294496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:55.336196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2d233-1871-4aae-9338-d3c69fa5583b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:55.379543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfdcb8-c51e-43e8-83dc-cfaa9a248246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:55.380196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfdcb8-c51e-43e8-83dc-cfaa9a248246 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:55.380504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfdcb8-c51e-43e8-83dc-cfaa9a248246 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:55.380829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfdcb8-c51e-43e8-83dc-cfaa9a248246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:55.391901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acfdcb8-c51e-43e8-83dc-cfaa9a248246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2acfdcb8-c51e-43e8-83dc-cfaa9a248246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:55.396969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:55.397936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:55.398798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 523/1048] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:55.405559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afbae1-8f20-412d-8074-15c1b222a8fd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:55.452389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ddfa7-21cf-4678-a92c-ac977abed7ca None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:55.452733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ddfa7-21cf-4678-a92c-ac977abed7ca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:55.452915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ddfa7-21cf-4678-a92c-ac977abed7ca None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:55.455019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 524/1049] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:55.459935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e52c-e8ae-4aa2-9b82-1069fab2503f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:55.460213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e52c-e8ae-4aa2-9b82-1069fab2503f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:55.460515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67e52c-e8ae-4aa2-9b82-1069fab2503f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:55.476465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:55.479291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8f67e52c-e8ae-4aa2-9b82-1069fab2503f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:55.481011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8f67e52c-e8ae-4aa2-9b82-1069fab2503f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:55.514543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afbae1-8f20-412d-8074-15c1b222a8fd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:55.549381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 525/1050] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:55.550307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:55.550928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:55.551199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:55.551458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:55.552110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:55.552432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:55.553772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:55.566981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 526/1051] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:55.568097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e52c-e8ae-4aa2-9b82-1069fab2503f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:55.624272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:55.668994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67e52c-e8ae-4aa2-9b82-1069fab2503f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:55.704252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:55.704890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:55.705193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:55.705461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] PATH_INFO: `/v3/users/033cfab749b84bbf9d556482785205ee` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:55.719125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] RBAC: Authorizing `identity:get_user(user_id=033cfab749b84bbf9d556482785205ee)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:55.720599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:55.727461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 526/1052] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:49:55 2022] GET /identity/v3/users/033cfab749b84bbf9d556482785205ee => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:55.728575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:55.812005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:55.836351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e43e-ff33-4e48-9522-49a1392f9e1b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:55.863507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:55.871671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:55.872331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:55.872589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:55.872846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:55.921804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdfba4-bb3a-4bba-927f-dd7229213556 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:55.936312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:55.948182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:55.949630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:55.962422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:55.963065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:55.963415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:55.963694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] PATH_INFO: `/v3/services/image` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:55.964475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:55.965692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:55.975103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:55.976662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:55.978065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 527/1053] 10.222.0.23 () {56 vars in 1105 bytes} [Mon Sep 5 16:49:55 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:55.982403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:56.055686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 527/1054] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 16:49:56.061539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:56.089729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:56.093401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a67ad-fcdd-4eef-ba89-271fb2fddd21 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:56.130176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:56.130822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:56.131179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:56.131447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:56.132068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] RBAC: Adding query filter params (name=image) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:56.132393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:56.133752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:56.144233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 528/1055] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:56.145283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:56.178643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e487e7-8888-4e27-9ab9-62425bbfc736 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:56.196516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:56.240928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:56.241493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:56.241769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:56.242037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:56.243473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] RBAC: Authorizing `identity:get_project(project_id=94ba9234922f4714abf3df343d594414)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:56.245380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:56.247662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 528/1056] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:49:56 2022] GET /identity/v3/projects/94ba9234922f4714abf3df343d594414 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:56.253338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:56.268215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8f00-32fd-40f0-8c0c-6814b33bca81 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:56.324286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:56.325186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:56.325602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:56.326032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:56.326823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] RBAC: Adding query filter params (type=image) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:56.327290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:56.328683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:56.336934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 529/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:56.342569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:56.397118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:56.403501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089bf17-d140-4965-aa40-699f23bbad0b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:56.440817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5d2ab-31dc-4fa4-8f1c-1d30810e2317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:56.441491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5d2ab-31dc-4fa4-8f1c-1d30810e2317 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:56.441674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5d2ab-31dc-4fa4-8f1c-1d30810e2317 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:56.441936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5d2ab-31dc-4fa4-8f1c-1d30810e2317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:56.442651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5d2ab-31dc-4fa4-8f1c-1d30810e2317 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=94ba9234922f4714abf3df343d594414, user.id=033cfab749b84bbf9d556482785205ee) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:56.442954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5d2ab-31dc-4fa4-8f1c-1d30810e2317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:56.444807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5d2ab-31dc-4fa4-8f1c-1d30810e2317 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:56.450878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 529/1058] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:49:56 2022] GET /identity/v3/role_assignments?user.id=033cfab749b84bbf9d556482785205ee&scope.project.id=94ba9234922f4714abf3df343d594414&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:56.469946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb8e4-1b08-4613-826e-09c470d937df None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:56.474039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:56.514719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1290b5a-f532-401d-8497-a4fa1402050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:56.515359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1290b5a-f532-401d-8497-a4fa1402050b None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:56.515707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1290b5a-f532-401d-8497-a4fa1402050b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:56.516049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1290b5a-f532-401d-8497-a4fa1402050b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:56.516848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1290b5a-f532-401d-8497-a4fa1402050b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:56.518486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1290b5a-f532-401d-8497-a4fa1402050b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:56.543475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 530/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:49:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:49:56.658433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9be19-074f-4f28-bc73-c0768aa785fb None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:56.658693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9be19-074f-4f28-bc73-c0768aa785fb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:56.658952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9be19-074f-4f28-bc73-c0768aa785fb None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:56.660598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 530/1060] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:56.666142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d10e15-c6b1-47c9-b540-e1bd51439243 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:56.666383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d10e15-c6b1-47c9-b540-e1bd51439243 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:56.666691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d10e15-c6b1-47c9-b540-e1bd51439243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:56.687548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-31d10e15-c6b1-47c9-b540-e1bd51439243 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:56.688800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-31d10e15-c6b1-47c9-b540-e1bd51439243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:56.767142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 531/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:49:57.037528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9be19-074f-4f28-bc73-c0768aa785fb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:57.146665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9be19-074f-4f28-bc73-c0768aa785fb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:57.179984 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:57.180640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:57.180916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:57.181170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:57.191430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:57.196215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:57.197125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:57.201153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 531/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Sep 5 16:49:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:57.203502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d10e15-c6b1-47c9-b540-e1bd51439243 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:57.287111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:57.305382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d10e15-c6b1-47c9-b540-e1bd51439243 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:57.340865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:57.341465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:57.341756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:57.342029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:57.342685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:57.342985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:57.344391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:57.353125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 532/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Sep 5 16:49:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:57.358973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:57.386616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:57.467056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba814190-eb87-4ee0-b193-ea84fba31cb5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:57.508099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:57.508768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:57.509040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:57.509305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:57.520443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:57.521926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:57.538502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:57.539512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:57.540855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:57.540855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:57.540855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:57.541293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 532/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:57.545448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:57.646881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe273f13-1d58-42a0-b255-7dba86e7908c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:57.684620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:57.685242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:57.685567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:57.685851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:57.686486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:57.686775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:57.688170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:57.704752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 533/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:57.706043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:57.776838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:57.810600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d084b0-d265-4610-b02a-7568cbb72b38 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:57.846215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:57.847128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:57.847540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:57.847939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:57.857789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:57.859320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:49:57.872317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:57.873302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:57.874242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 533/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:49:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:57.884269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:57.895720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:57.987444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717475aa-b31c-4660-b9df-9456af21960b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:58.022487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:58.023098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.023371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.023640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.024382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:58.024684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:58.026105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:58.043835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 534/1067] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:49:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:58.049640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:58.158119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de205812-ca73-457a-b8b5-f65ad47b33a1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:58.189036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:58.195120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26eab66-a065-445f-ae02-91d6c704042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:58.195752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26eab66-a065-445f-ae02-91d6c704042a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.196020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26eab66-a065-445f-ae02-91d6c704042a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.196317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26eab66-a065-445f-ae02-91d6c704042a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.207853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26eab66-a065-445f-ae02-91d6c704042a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:58.209326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26eab66-a065-445f-ae02-91d6c704042a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e26eab66-a065-445f-ae02-91d6c704042a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:58.223441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:58.224484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:58.225472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:58.225472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:58.225472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:58.225472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 534/1068] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:58.232306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:58.299652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:58.335810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a28d4-2a00-40ed-91ac-5ab60d0acc0c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:58.377018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:58.377018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.377018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.377018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.377395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:58.377619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:58.378922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:58.382293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cc11-0123-4b69-8660-0824193d57ae None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.382581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cc11-0123-4b69-8660-0824193d57ae None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.382857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739cc11-0123-4b69-8660-0824193d57ae None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.384447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 535/1069] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:49:58.391402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b632e-0642-4a2d-a203-9d80e9363ed6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.391687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b632e-0642-4a2d-a203-9d80e9363ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.392478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b632e-0642-4a2d-a203-9d80e9363ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.393817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 535/1070] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:58.399331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:58.412304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f32b632e-0642-4a2d-a203-9d80e9363ed6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:58.413482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f32b632e-0642-4a2d-a203-9d80e9363ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:58.489402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 536/1071] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:58.515058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98086da-661e-41bf-a0eb-e88c9a1912d4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:58.516587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4421c-711e-4231-968a-01c55a861e70 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.516874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4421c-711e-4231-968a-01c55a861e70 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.517160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4421c-711e-4231-968a-01c55a861e70 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.518794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 537/1072] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:49:58.523658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c160fd-6308-41b2-87b6-1dd1347ae096 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.523943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c160fd-6308-41b2-87b6-1dd1347ae096 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.524243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c160fd-6308-41b2-87b6-1dd1347ae096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.542907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-40c160fd-6308-41b2-87b6-1dd1347ae096 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:49:58.544071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:58.544219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-40c160fd-6308-41b2-87b6-1dd1347ae096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:49:58.553845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:58.554473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.554755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.555030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.562068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:58.563511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:49:58.574950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:58.575850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:58.576801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 536/1073] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:58.581907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:58.618473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 538/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:49:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:49:58.686529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd86088-ec8f-4051-87d3-be87ed5dd624 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:58.721152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:58.722051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.722514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.722972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.723844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:58.724381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:58.725997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:58.739274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 537/1075] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:58.745073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c160fd-6308-41b2-87b6-1dd1347ae096 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:58.763147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:58.793518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:58.814802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:58.851339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c160fd-6308-41b2-87b6-1dd1347ae096 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:58.873856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ac46a-56b1-4aff-8fad-cc6781cd7b9b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:58.887844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:58.888549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.888822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.889084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/c30e20ce4d63452095aedb60249f0cce/roles/59ee14375b7b400983f97480ca2fe124` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.891579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=c30e20ce4d63452095aedb60249f0cce, role_id=59ee14375b7b400983f97480ca2fe124)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:58.893156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:58.907980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 539/1076] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:58 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/c30e20ce4d63452095aedb60249f0cce/roles/59ee14375b7b400983f97480ca2fe124 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:49:58.909034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:58.913524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:58.914193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:58.914475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:58.914750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:58.929003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:49:58.933532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:58.934615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:49:58.935486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:49:58.938996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 538/1077] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:49:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:49:58.941310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:59.035381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0b206-e2bc-4bac-9e21-e2a0c497eb46 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:59.039088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:59.068586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ea41-b088-43a6-8c14-444604777dfb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:59.077654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:59.082042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:59.082778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:59.083056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:59.083331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:59.107743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:59.108390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:59.108675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:59.108940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:59.109604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:59.109909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:59.111347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:59.123344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 539/1078] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:49:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:49:59.129160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:59.155748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:59.157645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:59.232477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db935c2-d8f9-48a6-b52a-e48d44da07c1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:59.261193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 540/1079] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:49:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:49:59.267080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:59.270006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:59.271924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a80836-c429-435d-9807-29966f9d686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:59.272711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a80836-c429-435d-9807-29966f9d686d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:59.272998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a80836-c429-435d-9807-29966f9d686d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:59.273301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a80836-c429-435d-9807-29966f9d686d None admin] PATH_INFO: `/v3/users/033cfab749b84bbf9d556482785205ee` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:59.274579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a80836-c429-435d-9807-29966f9d686d None admin] RBAC: Authorizing `identity:get_user(user_id=033cfab749b84bbf9d556482785205ee)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:59.276368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a80836-c429-435d-9807-29966f9d686d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:59.277875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 540/1080] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:49:59 2022] GET /identity/v3/users/033cfab749b84bbf9d556482785205ee => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:59.283412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a80836-c429-435d-9807-29966f9d686d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:59.320926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:59.373231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acd9d9-6e5b-40ea-a277-8238782237f4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:59.401207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a80836-c429-435d-9807-29966f9d686d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:59.409541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:59.410184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:59.410460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:59.410739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:59.421025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:59.422470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:49:59.437873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:49:59.438842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:49:59.445285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:49:59.445285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:49:59.445285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:49:59.445461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:59.445461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:59.445461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:59.445461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:59.445739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 541/1081] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:49:59.445807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] RBAC: Authorizing `identity:get_project(project_id=94ba9234922f4714abf3df343d594414)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:59.445807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:59.445807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 541/1082] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:49:59 2022] GET /identity/v3/projects/94ba9234922f4714abf3df343d594414 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:59.449090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:59.449210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:49:59.510590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:59.557025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76a6e6-6648-4fcf-85c4-747ddd629cea None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:59.559418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a228af-4c0c-4514-9a33-871ed61eec94 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:49:59.561826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:49:59.595678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a243b2-de4f-4a52-a5fe-81a7a99b7ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:59.596030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e265e2-d9c0-4f75-85be-a3b3715f3dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:49:59.596717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a243b2-de4f-4a52-a5fe-81a7a99b7ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:59.596837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e265e2-d9c0-4f75-85be-a3b3715f3dd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:49:59.597001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e265e2-d9c0-4f75-85be-a3b3715f3dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:59.597103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a243b2-de4f-4a52-a5fe-81a7a99b7ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:49:59.597264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a243b2-de4f-4a52-a5fe-81a7a99b7ec3 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:59.597357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e265e2-d9c0-4f75-85be-a3b3715f3dd8 None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414/users/033cfab749b84bbf9d556482785205ee/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:49:59.597900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a243b2-de4f-4a52-a5fe-81a7a99b7ec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:49:59.598203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a243b2-de4f-4a52-a5fe-81a7a99b7ec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:59.599166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e265e2-d9c0-4f75-85be-a3b3715f3dd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ba9234922f4714abf3df343d594414, user_id=033cfab749b84bbf9d556482785205ee, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:49:59.599575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a243b2-de4f-4a52-a5fe-81a7a99b7ec3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:59.600709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e265e2-d9c0-4f75-85be-a3b3715f3dd8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:49:59.612830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 542/1083] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:49:59.623545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 542/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:49:59 2022] PUT /identity/v3/projects/94ba9234922f4714abf3df343d594414/users/033cfab749b84bbf9d556482785205ee/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:50:00.793986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68c9ad-862f-4b1d-9fbf-012190f4ca53 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:00.794536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68c9ad-862f-4b1d-9fbf-012190f4ca53 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:00.794536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68c9ad-862f-4b1d-9fbf-012190f4ca53 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:00.795813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 543/1085] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:00.801061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473330-48b4-4dfb-9ebb-bbdd9aea1d94 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:00.801481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473330-48b4-4dfb-9ebb-bbdd9aea1d94 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:00.801589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49473330-48b4-4dfb-9ebb-bbdd9aea1d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:00.820931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-49473330-48b4-4dfb-9ebb-bbdd9aea1d94 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:00.822109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49473330-48b4-4dfb-9ebb-bbdd9aea1d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:00.892179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 543/1086] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:01.158511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68c9ad-862f-4b1d-9fbf-012190f4ca53 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:01.270200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68c9ad-862f-4b1d-9fbf-012190f4ca53 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:01.287269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:01.307178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:01.307812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:01.308096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:01.308401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:01.318964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:01.323778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:01.324722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:01.325563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 544/1087] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Sep 5 16:50:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:01.331065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473330-48b4-4dfb-9ebb-bbdd9aea1d94 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:01.428983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:01.436612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49473330-48b4-4dfb-9ebb-bbdd9aea1d94 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:01.471378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b1c8-5ee1-4ccf-b20b-001540358a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:01.471976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b1c8-5ee1-4ccf-b20b-001540358a6d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:01.472291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b1c8-5ee1-4ccf-b20b-001540358a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:01.472566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b1c8-5ee1-4ccf-b20b-001540358a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:01.473222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b1c8-5ee1-4ccf-b20b-001540358a6d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:01.473523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b1c8-5ee1-4ccf-b20b-001540358a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:01.474901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b1c8-5ee1-4ccf-b20b-001540358a6d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:01.483099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 544/1088] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Sep 5 16:50:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:01.489294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:01.548613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec63f-dfe6-418f-ba72-de023837ab5b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:01.549000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec63f-dfe6-418f-ba72-de023837ab5b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:01.549276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec63f-dfe6-418f-ba72-de023837ab5b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:01.550637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 545/1089] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:01.555551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5b13f-4ca0-418e-b0cc-71afb6a97773 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:01.555833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5b13f-4ca0-418e-b0cc-71afb6a97773 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:01.556107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5b13f-4ca0-418e-b0cc-71afb6a97773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:01.574237 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4dd5b13f-4ca0-418e-b0cc-71afb6a97773 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:01.575426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4dd5b13f-4ca0-418e-b0cc-71afb6a97773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:01.592595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5038438-1c23-4f85-89fe-610b7159c358 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:01.629815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:01.630444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:01.630714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:01.630979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:01.640961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:01.642377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:01.644451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 546/1090] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:50:01.657349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:01.658422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:50:01.659297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:50:01.659297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:50:01.659297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:01.659297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 545/1091] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:01.664720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5b13f-4ca0-418e-b0cc-71afb6a97773 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:01.753796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:01.768876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5b13f-4ca0-418e-b0cc-71afb6a97773 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:01.803924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:01.804595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:01.804841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:01.805116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:01.805758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:01.806065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:01.807453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:01.819052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 547/1092] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:01.825012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:01.855953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:01.912493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:01.936764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482b7d1-7b80-4f86-bd53-1b031199bf47 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:01.973521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:01.974129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:01.974416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:01.974677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:01.982594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:01.984029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:50:01.995190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:01.996341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:01.997327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 546/1093] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Sep 5 16:50:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:02.002311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.017072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f472a5f1-8a11-4119-a80e-5cb8cecbc0a7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.051810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.052475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.052749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.053021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.063551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:02.068501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:02.069474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:50:02.070309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 548/1094] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:02.075545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.085048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.089217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.102497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0380895-7742-4641-92e9-edbdf401b9cd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.136959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.137908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.138253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.138514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.139240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:02.139538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.140976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.157563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 547/1095] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:02.163902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.176951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e55000-7517-419b-a479-a84d3e299e15 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.211365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.211981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.212295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.212568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.213231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:02.213531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.214930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.224710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 549/1096] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:02.230564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.266579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85753-1f2f-4e7e-a951-4eeaee879753 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.298178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.300748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.301367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.301650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.301932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.317379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.334034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51d39a-e846-4065-9f16-c54dc4250820 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.369636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.370272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.370545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.370812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] PATH_INFO: `/v3/users/033cfab749b84bbf9d556482785205ee` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.373073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] RBAC: Authorizing `identity:get_user(user_id=033cfab749b84bbf9d556482785205ee)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.374515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.376106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 550/1097] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/users/033cfab749b84bbf9d556482785205ee => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:02.379950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.381428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.382004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.479390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 548/1098] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 16:50:02.482013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c810ad1-e973-405f-b743-eea01b62a7ce None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.485493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.516797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.517528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.517900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.518175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.521545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.530554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.586613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77617c95-6636-4176-875e-11622a02dc20 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.591067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.592574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.622986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.623608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.623879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.624165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.634070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.635658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:50:02.650470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:02.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:50:02.652419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:50:02.652419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:50:02.652419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:02.652419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 549/1099] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:02.657300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.696316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 551/1100] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 16:50:02.702271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.734892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.755499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.759862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1522ff8-e3a7-4eef-919a-fd573007997a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.795031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.795630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.795905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.796179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.796840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:02.797131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.798492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.800112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977382f-36d8-46f2-82af-c8bff0a01ace None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.810742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 550/1101] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:02.817045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.834790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.835432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.835708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.835981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.837152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] RBAC: Authorizing `identity:get_project(project_id=94ba9234922f4714abf3df343d594414)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.838528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.840390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 552/1102] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/projects/94ba9234922f4714abf3df343d594414 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:02.846221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.916571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f3ff7-e73c-4a05-91c0-1c4fd70bfb2a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.948661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a47909-e771-443e-bf2f-dcadb4f60898 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:02.949349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.949964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.950240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.950502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.951096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:02.957962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.959456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:50:02.971331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:02.972418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:02.973790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 551/1103] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:50:02.978248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:02.987713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:02.988464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:02.988739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:02.989003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:02.989724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=94ba9234922f4714abf3df343d594414, user.id=033cfab749b84bbf9d556482785205ee) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:02.990031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:02.991440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:02.997398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 553/1104] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/role_assignments?user.id=033cfab749b84bbf9d556482785205ee&scope.project.id=94ba9234922f4714abf3df343d594414&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:02.997792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:03.087657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79ae07-0806-4211-86a8-08682a25625c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:03.126587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539a05d-d99d-4ee3-8f94-8897722b3dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:03.127410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539a05d-d99d-4ee3-8f94-8897722b3dcf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:03.128018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539a05d-d99d-4ee3-8f94-8897722b3dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:03.128018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539a05d-d99d-4ee3-8f94-8897722b3dcf None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:03.128818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539a05d-d99d-4ee3-8f94-8897722b3dcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:03.129125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539a05d-d99d-4ee3-8f94-8897722b3dcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:03.130584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539a05d-d99d-4ee3-8f94-8897722b3dcf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:03.143605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 552/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:03.149918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:03.251455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775a031-f71a-4118-81a9-8ee96e50dbb9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:03.288887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fb90c-d06b-4462-b186-a0d63b180f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:03.289501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fb90c-d06b-4462-b186-a0d63b180f3c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:03.289780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fb90c-d06b-4462-b186-a0d63b180f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:03.290047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fb90c-d06b-4462-b186-a0d63b180f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:03.290765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fb90c-d06b-4462-b186-a0d63b180f3c None admin] RBAC: Adding query filter params (role.id=59ee14375b7b400983f97480ca2fe124, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=c30e20ce4d63452095aedb60249f0cce) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:03.291056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fb90c-d06b-4462-b186-a0d63b180f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:03.292518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fb90c-d06b-4462-b186-a0d63b180f3c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:03.298338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 554/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:03 2022] GET /identity/v3/role_assignments?user.id=c30e20ce4d63452095aedb60249f0cce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=59ee14375b7b400983f97480ca2fe124 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:04.905707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3a65-70ad-4050-9bec-6f6e20ea6345 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:04.906320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3a65-70ad-4050-9bec-6f6e20ea6345 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:04.906320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3a65-70ad-4050-9bec-6f6e20ea6345 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:04.907595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 553/1107] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:04.913935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3f259-d4de-46b6-bdb4-363377161460 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:04.914760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3f259-d4de-46b6-bdb4-363377161460 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:04.914910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3f259-d4de-46b6-bdb4-363377161460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:04.938364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-75a3f259-d4de-46b6-bdb4-363377161460 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:04.939773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-75a3f259-d4de-46b6-bdb4-363377161460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:04.969230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:05.016352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 555/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:05.300461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3a65-70ad-4050-9bec-6f6e20ea6345 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:05.395704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:05.418969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3a65-70ad-4050-9bec-6f6e20ea6345 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:05.453872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ceb7d-2870-4428-9207-4758c468d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:05.454489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ceb7d-2870-4428-9207-4758c468d653 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:05.454757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ceb7d-2870-4428-9207-4758c468d653 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:05.455088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ceb7d-2870-4428-9207-4758c468d653 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:05.456604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ceb7d-2870-4428-9207-4758c468d653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:05.458053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ceb7d-2870-4428-9207-4758c468d653 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:05.459663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 554/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:50:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:05.468852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3f259-d4de-46b6-bdb4-363377161460 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:05.578374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3f259-d4de-46b6-bdb4-363377161460 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:05.619729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d1f56-c3d4-489d-90e4-337fd8515cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:05.620380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d1f56-c3d4-489d-90e4-337fd8515cf8 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:05.620662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d1f56-c3d4-489d-90e4-337fd8515cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:05.620938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d1f56-c3d4-489d-90e4-337fd8515cf8 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:05.621588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d1f56-c3d4-489d-90e4-337fd8515cf8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:05.623041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d1f56-c3d4-489d-90e4-337fd8515cf8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:05.646547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 556/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:50:05 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:50:07.576330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ebc5-c0c8-4986-ac20-478f37edf336 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:07.577166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ebc5-c0c8-4986-ac20-478f37edf336 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:07.577166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5ebc5-c0c8-4986-ac20-478f37edf336 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:07.579337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 555/1111] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:07.584956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd36884-8f87-4bef-a0ee-72d60d4ab7cd None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:07.585397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd36884-8f87-4bef-a0ee-72d60d4ab7cd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:07.585491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd36884-8f87-4bef-a0ee-72d60d4ab7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:07.606216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2cd36884-8f87-4bef-a0ee-72d60d4ab7cd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:07.607389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2cd36884-8f87-4bef-a0ee-72d60d4ab7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:07.621667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:07.680273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 557/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:07.944941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5ebc5-c0c8-4986-ac20-478f37edf336 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:08.059428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5ebc5-c0c8-4986-ac20-478f37edf336 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:08.096446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:08.097060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:08.097333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:08.097608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] PATH_INFO: `/v3/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:08.099054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] RBAC: Authorizing `identity:get_role(role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:08.100450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:08.101808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 556/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:07 2022] GET /identity/v3/roles/18b82b06d87e4c93bef42bca94fc658d => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:08.111940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd36884-8f87-4bef-a0ee-72d60d4ab7cd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:08.217029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd36884-8f87-4bef-a0ee-72d60d4ab7cd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:08.252670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:08.253291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:08.253569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:08.253846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] PATH_INFO: `/v3/users/94b63ed63a174af19ead8140ab551cbc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:08.267231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] RBAC: Authorizing `identity:get_user(user_id=94b63ed63a174af19ead8140ab551cbc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:08.268862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:08.270450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 558/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:08 2022] GET /identity/v3/users/94b63ed63a174af19ead8140ab551cbc => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:08.279682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:08.297959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:08.387147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a2eef-43d6-49ff-9ee4-e8eb37829289 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:08.421490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:08.422089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:08.422360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:08.422630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:08.492431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:08.493848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:08.510538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:08.605455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 557/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:50:08.611947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:08.623215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:08.724647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9cc94-0579-4238-b554-02a86dc56386 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:08.764560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef14299-aa8b-46c1-a3c4-fdc60faac921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:08.765120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14299-aa8b-46c1-a3c4-fdc60faac921 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:08.765383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14299-aa8b-46c1-a3c4-fdc60faac921 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:08.765650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef14299-aa8b-46c1-a3c4-fdc60faac921 None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:08.766807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef14299-aa8b-46c1-a3c4-fdc60faac921 None admin] RBAC: Authorizing `identity:get_project(project_id=94ba9234922f4714abf3df343d594414)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:08.768353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef14299-aa8b-46c1-a3c4-fdc60faac921 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:08.769969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 559/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:08 2022] GET /identity/v3/projects/94ba9234922f4714abf3df343d594414 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:08.779651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:08.894228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58aa98-125c-483b-9282-703ecb0b4cc6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:08.908737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:08.930578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe60af6-2e33-433b-ba66-05d785076f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:08.931168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60af6-2e33-433b-ba66-05d785076f8c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:08.931440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60af6-2e33-433b-ba66-05d785076f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:08.931706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60af6-2e33-433b-ba66-05d785076f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:08.932450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe60af6-2e33-433b-ba66-05d785076f8c None admin] RBAC: Adding query filter params (role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=94ba9234922f4714abf3df343d594414, user.id=94b63ed63a174af19ead8140ab551cbc) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:08.932750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe60af6-2e33-433b-ba66-05d785076f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:08.934153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe60af6-2e33-433b-ba66-05d785076f8c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:08.939653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 558/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:08 2022] GET /identity/v3/role_assignments?user.id=94b63ed63a174af19ead8140ab551cbc&scope.project.id=94ba9234922f4714abf3df343d594414&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:10.774651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52747-3fd0-4ee8-9333-e568200da1ed None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:10.775175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52747-3fd0-4ee8-9333-e568200da1ed None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:10.775175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b52747-3fd0-4ee8-9333-e568200da1ed None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:10.776490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 560/1118] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:10.783276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157d8ef-7581-4f1b-9530-8a19d3c7c1c6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:10.783728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157d8ef-7581-4f1b-9530-8a19d3c7c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:10.783831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f157d8ef-7581-4f1b-9530-8a19d3c7c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:10.808815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f157d8ef-7581-4f1b-9530-8a19d3c7c1c6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:10.810020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f157d8ef-7581-4f1b-9530-8a19d3c7c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:10.889271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 559/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:11.156480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b52747-3fd0-4ee8-9333-e568200da1ed None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:11.257440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:11.284823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b52747-3fd0-4ee8-9333-e568200da1ed None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:11.320020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:11.320670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:11.320949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:11.321226 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] PATH_INFO: `/v3/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:11.322731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] RBAC: Authorizing `identity:get_role(role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:11.324113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:11.325475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 561/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:11 2022] GET /identity/v3/roles/18b82b06d87e4c93bef42bca94fc658d => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:11.332220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157d8ef-7581-4f1b-9530-8a19d3c7c1c6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:11.441813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f157d8ef-7581-4f1b-9530-8a19d3c7c1c6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:11.450772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:11.477630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:11.478255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:11.478531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:11.478802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] PATH_INFO: `/v3/users/94b63ed63a174af19ead8140ab551cbc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:11.480042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] RBAC: Authorizing `identity:get_user(user_id=94b63ed63a174af19ead8140ab551cbc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:11.481822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:11.483418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 560/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:11 2022] GET /identity/v3/users/94b63ed63a174af19ead8140ab551cbc => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:11.490159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:11.606520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba2012-b7f1-4b9c-8a3a-c7e382150515 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:11.646594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eab82-bf9b-46d6-8035-cf9dfeb6ebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:11.647591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eab82-bf9b-46d6-8035-cf9dfeb6ebbf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:11.648051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eab82-bf9b-46d6-8035-cf9dfeb6ebbf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:11.648581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eab82-bf9b-46d6-8035-cf9dfeb6ebbf None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:11.650638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1eab82-bf9b-46d6-8035-cf9dfeb6ebbf None admin] RBAC: Authorizing `identity:get_project(project_id=94ba9234922f4714abf3df343d594414)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:11.653056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1eab82-bf9b-46d6-8035-cf9dfeb6ebbf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:11.655682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 562/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:11 2022] GET /identity/v3/projects/94ba9234922f4714abf3df343d594414 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:11.665528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:11.773283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f41de-4f7f-4938-a739-1543107dc603 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:11.808322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4119d6-7c12-4c20-b74c-f773f9d69d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:11.808929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4119d6-7c12-4c20-b74c-f773f9d69d4d None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:11.809222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4119d6-7c12-4c20-b74c-f773f9d69d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:11.809487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4119d6-7c12-4c20-b74c-f773f9d69d4d None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414/users/94b63ed63a174af19ead8140ab551cbc/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:11.812002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4119d6-7c12-4c20-b74c-f773f9d69d4d None admin] RBAC: Authorizing `identity:create_grant(project_id=94ba9234922f4714abf3df343d594414, user_id=94b63ed63a174af19ead8140ab551cbc, role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:11.813446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4119d6-7c12-4c20-b74c-f773f9d69d4d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:11.828901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 561/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:50:11 2022] PUT /identity/v3/projects/94ba9234922f4714abf3df343d594414/users/94b63ed63a174af19ead8140ab551cbc/roles/18b82b06d87e4c93bef42bca94fc658d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:50:13.775722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e06fc-92a0-49f1-a14b-b3b23da22cef None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:13.776789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e06fc-92a0-49f1-a14b-b3b23da22cef None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:13.776789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e06fc-92a0-49f1-a14b-b3b23da22cef None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:13.778683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 563/1124] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:13.786788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7055f39-913b-4754-81f0-ff4f22d8ee03 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:13.787250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7055f39-913b-4754-81f0-ff4f22d8ee03 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:13.787344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7055f39-913b-4754-81f0-ff4f22d8ee03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:13.807464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e7055f39-913b-4754-81f0-ff4f22d8ee03 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:13.808742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e7055f39-913b-4754-81f0-ff4f22d8ee03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:13.840881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:13.881020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 562/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:50:14.152800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e06fc-92a0-49f1-a14b-b3b23da22cef None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:14.167145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:14.263531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e06fc-92a0-49f1-a14b-b3b23da22cef None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:14.300445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:14.301065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:14.301344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:14.301614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] PATH_INFO: `/v3/roles/18b82b06d87e4c93bef42bca94fc658d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:14.303114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] RBAC: Authorizing `identity:get_role(role_id=18b82b06d87e4c93bef42bca94fc658d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:14.304526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:14.305857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 564/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:14 2022] GET /identity/v3/roles/18b82b06d87e4c93bef42bca94fc658d => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:14.312182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7055f39-913b-4754-81f0-ff4f22d8ee03 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:14.435753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7055f39-913b-4754-81f0-ff4f22d8ee03 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:14.473081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:14.473705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:14.473983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:14.474258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] PATH_INFO: `/v3/users/94b63ed63a174af19ead8140ab551cbc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:14.475422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] RBAC: Authorizing `identity:get_user(user_id=94b63ed63a174af19ead8140ab551cbc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:14.476896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:14.478900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 563/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:14 2022] GET /identity/v3/users/94b63ed63a174af19ead8140ab551cbc => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:14.485416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:14.584117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:14.619292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef54e0-d538-4c16-b99c-ebf0d4437572 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:14.668436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:14.669366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:14.669855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:14.670276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:14.742027 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:14.743405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:14.820296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:14.842822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 565/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:50:14.852077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:14.888527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:14.968630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2bfe8-74b6-49a4-8c99-226fa0e68b2a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:15.007223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b81f8-70e8-499b-bb8a-c18e777f0de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:15.007969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b81f8-70e8-499b-bb8a-c18e777f0de3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:15.008360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b81f8-70e8-499b-bb8a-c18e777f0de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:15.008709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b81f8-70e8-499b-bb8a-c18e777f0de3 None admin] PATH_INFO: `/v3/projects/94ba9234922f4714abf3df343d594414` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:15.010550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b81f8-70e8-499b-bb8a-c18e777f0de3 None admin] RBAC: Authorizing `identity:get_project(project_id=94ba9234922f4714abf3df343d594414)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:15.012809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b81f8-70e8-499b-bb8a-c18e777f0de3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:15.015250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 564/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:14 2022] GET /identity/v3/projects/94ba9234922f4714abf3df343d594414 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:15.021788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:15.126374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097fcc0-7028-4006-9a91-f0f9354483f6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:15.165618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1d1be-12d1-4f8a-9aaa-3cc3465274a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:15.166230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1d1be-12d1-4f8a-9aaa-3cc3465274a7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:15.166509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1d1be-12d1-4f8a-9aaa-3cc3465274a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:15.166781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1d1be-12d1-4f8a-9aaa-3cc3465274a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:15.167511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1d1be-12d1-4f8a-9aaa-3cc3465274a7 None admin] RBAC: Adding query filter params (role.id=18b82b06d87e4c93bef42bca94fc658d, scope.project.id=94ba9234922f4714abf3df343d594414, user.id=94b63ed63a174af19ead8140ab551cbc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:15.167826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1d1be-12d1-4f8a-9aaa-3cc3465274a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:15.169348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1d1be-12d1-4f8a-9aaa-3cc3465274a7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:15.175771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 566/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:15 2022] GET /identity/v3/role_assignments?user.id=94b63ed63a174af19ead8140ab551cbc&scope.project.id=94ba9234922f4714abf3df343d594414&role.id=18b82b06d87e4c93bef42bca94fc658d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:17.054275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3b392-fedc-442a-b48b-b3510b5ce72c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:17.054843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3b392-fedc-442a-b48b-b3510b5ce72c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:17.054843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3b392-fedc-442a-b48b-b3510b5ce72c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:17.056167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 565/1131] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:17.062425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e51cb-7722-4ef2-96ca-ac4be15f05b5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:17.063182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e51cb-7722-4ef2-96ca-ac4be15f05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:17.063332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e51cb-7722-4ef2-96ca-ac4be15f05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:17.088782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8a4e51cb-7722-4ef2-96ca-ac4be15f05b5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:17.089961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8a4e51cb-7722-4ef2-96ca-ac4be15f05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:17.130747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:17.175660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 567/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:17.452026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3b392-fedc-442a-b48b-b3510b5ce72c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:17.568912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3b392-fedc-442a-b48b-b3510b5ce72c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:17.570587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:17.606559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f3f39e-ab94-4a22-8fdd-3a00537f8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:17.607172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f39e-ab94-4a22-8fdd-3a00537f8646 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:17.607449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f39e-ab94-4a22-8fdd-3a00537f8646 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:17.607710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f3f39e-ab94-4a22-8fdd-3a00537f8646 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:17.608860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3f39e-ab94-4a22-8fdd-3a00537f8646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:17.610276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f3f39e-ab94-4a22-8fdd-3a00537f8646 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:17.611938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 566/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:50:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:17.618101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e51cb-7722-4ef2-96ca-ac4be15f05b5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:17.723522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e51cb-7722-4ef2-96ca-ac4be15f05b5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:17.758992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd2365-5b11-45ba-9013-14320d693864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:17.759588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd2365-5b11-45ba-9013-14320d693864 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:17.759861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd2365-5b11-45ba-9013-14320d693864 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:17.760129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd2365-5b11-45ba-9013-14320d693864 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:17.760812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd2365-5b11-45ba-9013-14320d693864 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:17.762252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd2365-5b11-45ba-9013-14320d693864 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:17.793411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 568/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:50:17 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 16:50:19.690079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421616ee-d350-4485-a824-c8daed5412cd None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:19.690916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421616ee-d350-4485-a824-c8daed5412cd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:19.690916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421616ee-d350-4485-a824-c8daed5412cd None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:19.693060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 567/1135] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:19.698451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877dfa1-b845-4641-a0f1-22265106b0aa None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:19.698896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877dfa1-b845-4641-a0f1-22265106b0aa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:19.699005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877dfa1-b845-4641-a0f1-22265106b0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:19.719735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1877dfa1-b845-4641-a0f1-22265106b0aa None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:19.721270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1877dfa1-b845-4641-a0f1-22265106b0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:19.740034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:19.803830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 569/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:20.083110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421616ee-d350-4485-a824-c8daed5412cd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:20.206640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421616ee-d350-4485-a824-c8daed5412cd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:20.243646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974c38d-838a-4cdc-8f1c-fd8fb153c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:20.244289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974c38d-838a-4cdc-8f1c-fd8fb153c927 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:20.244565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974c38d-838a-4cdc-8f1c-fd8fb153c927 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:20.244832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974c38d-838a-4cdc-8f1c-fd8fb153c927 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:20.245989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974c38d-838a-4cdc-8f1c-fd8fb153c927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:20.247424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974c38d-838a-4cdc-8f1c-fd8fb153c927 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:20.249029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 568/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:50:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:20.255121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877dfa1-b845-4641-a0f1-22265106b0aa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:20.374239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877dfa1-b845-4641-a0f1-22265106b0aa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:20.412941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d4d6f-d041-4ddc-80ce-2082c8e8f2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:20.413545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4d6f-d041-4ddc-80ce-2082c8e8f2e4 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:20.413822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4d6f-d041-4ddc-80ce-2082c8e8f2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:20.414096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d4d6f-d041-4ddc-80ce-2082c8e8f2e4 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:20.414738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d4d6f-d041-4ddc-80ce-2082c8e8f2e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:20.416194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d4d6f-d041-4ddc-80ce-2082c8e8f2e4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:20.446104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 570/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:50:20 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:50:22.359522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c2f7d-bcef-4702-9f4e-67b80dfa0700 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:22.360423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c2f7d-bcef-4702-9f4e-67b80dfa0700 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:22.360423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c2f7d-bcef-4702-9f4e-67b80dfa0700 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:22.362746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 569/1139] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:22.368840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf55a38-c821-46c7-8e10-23bcf6e64bf1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:22.369542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf55a38-c821-46c7-8e10-23bcf6e64bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:22.369998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf55a38-c821-46c7-8e10-23bcf6e64bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:22.390885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8cf55a38-c821-46c7-8e10-23bcf6e64bf1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:22.392231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8cf55a38-c821-46c7-8e10-23bcf6e64bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:22.395058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:22.461970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 571/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:22.729731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c2f7d-bcef-4702-9f4e-67b80dfa0700 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:22.775171 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:22.836999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c2f7d-bcef-4702-9f4e-67b80dfa0700 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:22.873351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:22.873962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:22.874233 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:22.874502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:22.885712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:22.890427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:22.891562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:22.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 570/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:50:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:22.897691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf55a38-c821-46c7-8e10-23bcf6e64bf1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:22.998814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf55a38-c821-46c7-8e10-23bcf6e64bf1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:23.035196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:23.035793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:23.036066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:23.036360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:23.037012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:23.037310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:23.038685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:23.046812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 572/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:50:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:23.052987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:23.163272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:23.166982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68792aa-f476-4fb0-8574-c186df47957c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:23.201259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:23.201871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:23.202143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:23.202411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] PATH_INFO: `/v3/users/2f64c65d670d4e48a6f32b72603ac1a7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:23.215945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] RBAC: Authorizing `identity:get_user(user_id=2f64c65d670d4e48a6f32b72603ac1a7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:23.217413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:23.219000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 571/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:23 2022] GET /identity/v3/users/2f64c65d670d4e48a6f32b72603ac1a7 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:23.225318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:23.228828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:23.332830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc5469-299f-43d2-b8f4-565961544a40 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:23.368309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:23.368903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:23.369451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:23.369451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:23.440120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:23.441598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:23.443425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:23.550730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 573/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:50:23.557217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:23.664682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb98bfe-adf4-4b74-9bf0-a853353b74f1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:23.700682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60095afa-7b93-472f-97b6-8db12628fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:23.701296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60095afa-7b93-472f-97b6-8db12628fa68 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:23.701575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60095afa-7b93-472f-97b6-8db12628fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:23.701839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60095afa-7b93-472f-97b6-8db12628fa68 None admin] PATH_INFO: `/v3/projects/6deda0000c114f53978818ee7285ca2c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:23.702955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60095afa-7b93-472f-97b6-8db12628fa68 None admin] RBAC: Authorizing `identity:get_project(project_id=6deda0000c114f53978818ee7285ca2c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:23.704498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60095afa-7b93-472f-97b6-8db12628fa68 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:23.706144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 572/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:23 2022] GET /identity/v3/projects/6deda0000c114f53978818ee7285ca2c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:23.715569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:23.827185 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106deb-a098-43f4-b545-d60302bee987 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:23.828942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:23.867298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded2121-2592-4193-9430-ce3ee52c007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:23.867900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded2121-2592-4193-9430-ce3ee52c007e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:23.868200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded2121-2592-4193-9430-ce3ee52c007e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:23.868476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded2121-2592-4193-9430-ce3ee52c007e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:23.869190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded2121-2592-4193-9430-ce3ee52c007e None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=6deda0000c114f53978818ee7285ca2c, user.id=2f64c65d670d4e48a6f32b72603ac1a7) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:23.869496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded2121-2592-4193-9430-ce3ee52c007e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:23.870929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded2121-2592-4193-9430-ce3ee52c007e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:23.876405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 574/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:23 2022] GET /identity/v3/role_assignments?user.id=2f64c65d670d4e48a6f32b72603ac1a7&scope.project.id=6deda0000c114f53978818ee7285ca2c&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:25.713643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a0fd-04a9-4bb9-b5d2-2e412cab7b9c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:25.714451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a0fd-04a9-4bb9-b5d2-2e412cab7b9c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:25.714451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a0fd-04a9-4bb9-b5d2-2e412cab7b9c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:25.716570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 573/1147] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:25.721993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a483aaf-b4ac-4458-b6c7-99d4a7aea53e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:25.722417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a483aaf-b4ac-4458-b6c7-99d4a7aea53e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:25.722531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a483aaf-b4ac-4458-b6c7-99d4a7aea53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:25.742362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7a483aaf-b4ac-4458-b6c7-99d4a7aea53e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:25.743547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7a483aaf-b4ac-4458-b6c7-99d4a7aea53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:25.812728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 575/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:26.077970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6a0fd-04a9-4bb9-b5d2-2e412cab7b9c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:26.118734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:26.213786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6a0fd-04a9-4bb9-b5d2-2e412cab7b9c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:26.248933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:26.249548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:26.249823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:26.250088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:26.259953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:26.264684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:26.266230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:26.268932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 574/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:26.274451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a483aaf-b4ac-4458-b6c7-99d4a7aea53e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:26.383467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:26.390572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a483aaf-b4ac-4458-b6c7-99d4a7aea53e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:26.429173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:26.429777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:26.430029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:26.430396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:26.431048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:26.431342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:26.432755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:26.441048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 576/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:26.449701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:26.547205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:26.568741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b05818-ef58-4c36-bda5-3691fe73e5b1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:26.610926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:26.611642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:26.611926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:26.612198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] PATH_INFO: `/v3/users/2f64c65d670d4e48a6f32b72603ac1a7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:26.613444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] RBAC: Authorizing `identity:get_user(user_id=2f64c65d670d4e48a6f32b72603ac1a7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:26.614893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:26.616572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 575/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:26 2022] GET /identity/v3/users/2f64c65d670d4e48a6f32b72603ac1a7 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:26.623340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:26.772461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9debed-e112-489f-bbc1-b3f172d6d0d0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:26.814341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80978a8-a24e-461a-9143-c0e275b1107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:26.814984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80978a8-a24e-461a-9143-c0e275b1107d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:26.815261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80978a8-a24e-461a-9143-c0e275b1107d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:26.815694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80978a8-a24e-461a-9143-c0e275b1107d None admin] PATH_INFO: `/v3/projects/6deda0000c114f53978818ee7285ca2c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:26.817246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80978a8-a24e-461a-9143-c0e275b1107d None admin] RBAC: Authorizing `identity:get_project(project_id=6deda0000c114f53978818ee7285ca2c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:26.818737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80978a8-a24e-461a-9143-c0e275b1107d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:26.820452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 577/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:26 2022] GET /identity/v3/projects/6deda0000c114f53978818ee7285ca2c => generated 311 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:26.830317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:26.946140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143ea3f-290a-49d9-b92d-9185256d7d8e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:26.987617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916d69c-98d3-429f-81b2-d371368e28c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:26.988308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916d69c-98d3-429f-81b2-d371368e28c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:26.988587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916d69c-98d3-429f-81b2-d371368e28c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:26.988859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916d69c-98d3-429f-81b2-d371368e28c9 None admin] PATH_INFO: `/v3/projects/6deda0000c114f53978818ee7285ca2c/users/2f64c65d670d4e48a6f32b72603ac1a7/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:26.990841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916d69c-98d3-429f-81b2-d371368e28c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6deda0000c114f53978818ee7285ca2c, user_id=2f64c65d670d4e48a6f32b72603ac1a7, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:26.992293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916d69c-98d3-429f-81b2-d371368e28c9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:27.013318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 576/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:50:26 2022] PUT /identity/v3/projects/6deda0000c114f53978818ee7285ca2c/users/2f64c65d670d4e48a6f32b72603ac1a7/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:50:28.885183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5172-79a6-4c04-bf0a-4c7a19d18177 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:28.885736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5172-79a6-4c04-bf0a-4c7a19d18177 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:28.885736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b5172-79a6-4c04-bf0a-4c7a19d18177 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:28.887078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 578/1154] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:28.893487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e3b8-4fc2-4645-a1ad-6e575782bb05 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:28.894228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e3b8-4fc2-4645-a1ad-6e575782bb05 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:28.894396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906e3b8-4fc2-4645-a1ad-6e575782bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:28.924459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5906e3b8-4fc2-4645-a1ad-6e575782bb05 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:28.925653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5906e3b8-4fc2-4645-a1ad-6e575782bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:28.933900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:28.999552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 577/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:29.264284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5172-79a6-4c04-bf0a-4c7a19d18177 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:29.294978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:29.381168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b5172-79a6-4c04-bf0a-4c7a19d18177 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:29.417592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:29.418211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:29.418486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:29.418754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:29.429614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:29.434378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:29.435470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:29.436570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:50:29.440343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 579/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:29.441513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906e3b8-4fc2-4645-a1ad-6e575782bb05 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:29.545034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906e3b8-4fc2-4645-a1ad-6e575782bb05 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:29.582232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:29.582826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:29.583103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:29.583377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:29.584034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:29.584364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:29.585734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:29.593886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 578/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:29.603255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:29.693523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:29.709203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072144e4-a78a-4b54-bfcd-82d5e20b5ede None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:29.744775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:29.745403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:29.745679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:29.745942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] PATH_INFO: `/v3/users/2f64c65d670d4e48a6f32b72603ac1a7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:29.747197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] RBAC: Authorizing `identity:get_user(user_id=2f64c65d670d4e48a6f32b72603ac1a7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:29.748692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:29.750251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 580/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:29 2022] GET /identity/v3/users/2f64c65d670d4e48a6f32b72603ac1a7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:29.756514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:29.760921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:29.862059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ecf04-75fe-485d-aa9d-a18f95ee9431 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:29.904020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:29.904722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:29.904911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:29.905173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:29.979276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:29.981066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:29.983741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:30.095439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 579/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:50:30.104755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:30.221967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdde52-1aa2-460d-907e-7db340bf5509 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:30.258347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf90daf-bef4-4a6a-ad81-a76cdb10da3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:30.258973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf90daf-bef4-4a6a-ad81-a76cdb10da3d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:30.259250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf90daf-bef4-4a6a-ad81-a76cdb10da3d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:30.259523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf90daf-bef4-4a6a-ad81-a76cdb10da3d None admin] PATH_INFO: `/v3/projects/6deda0000c114f53978818ee7285ca2c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:30.260796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf90daf-bef4-4a6a-ad81-a76cdb10da3d None admin] RBAC: Authorizing `identity:get_project(project_id=6deda0000c114f53978818ee7285ca2c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:30.262248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf90daf-bef4-4a6a-ad81-a76cdb10da3d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:30.263909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 581/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:30 2022] GET /identity/v3/projects/6deda0000c114f53978818ee7285ca2c => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:30.270289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:30.379857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e9fe0-e7e6-4995-b4de-8844ee60ad00 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:30.382307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:30.416465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3320ae1-e488-4005-9804-f29e67094d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:30.417066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320ae1-e488-4005-9804-f29e67094d9c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:30.417344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320ae1-e488-4005-9804-f29e67094d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:30.417611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3320ae1-e488-4005-9804-f29e67094d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:30.418332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3320ae1-e488-4005-9804-f29e67094d9c None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=6deda0000c114f53978818ee7285ca2c, user.id=2f64c65d670d4e48a6f32b72603ac1a7) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:30.418637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3320ae1-e488-4005-9804-f29e67094d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:30.420091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3320ae1-e488-4005-9804-f29e67094d9c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:30.426090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 580/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:30 2022] GET /identity/v3/role_assignments?user.id=2f64c65d670d4e48a6f32b72603ac1a7&scope.project.id=6deda0000c114f53978818ee7285ca2c&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:32.311413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724039d-b85c-4b05-b58f-0c6d043671f5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:32.311978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724039d-b85c-4b05-b58f-0c6d043671f5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:32.311978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724039d-b85c-4b05-b58f-0c6d043671f5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:32.313325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 582/1162] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:32.319649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7064a-6ba0-463e-90d4-72dfa7a42b31 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:32.320414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7064a-6ba0-463e-90d4-72dfa7a42b31 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:32.320560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7064a-6ba0-463e-90d4-72dfa7a42b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:32.351781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dce7064a-6ba0-463e-90d4-72dfa7a42b31 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:32.353597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dce7064a-6ba0-463e-90d4-72dfa7a42b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:32.435797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 581/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:32.702032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724039d-b85c-4b05-b58f-0c6d043671f5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:32.731547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:32.827310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724039d-b85c-4b05-b58f-0c6d043671f5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:32.865579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:32.866193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:32.866477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:32.866755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:32.948762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:32.951079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:32.982677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:33.066276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 583/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:50:33.075322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7064a-6ba0-463e-90d4-72dfa7a42b31 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:33.189167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:33.193910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7064a-6ba0-463e-90d4-72dfa7a42b31 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:33.232497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fb836-4d15-4bcf-b72b-ec8c7d1026ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:33.233177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fb836-4d15-4bcf-b72b-ec8c7d1026ef None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:33.233360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fb836-4d15-4bcf-b72b-ec8c7d1026ef None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:33.233658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fb836-4d15-4bcf-b72b-ec8c7d1026ef None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:33.240877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fb836-4d15-4bcf-b72b-ec8c7d1026ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:33.242305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fb836-4d15-4bcf-b72b-ec8c7d1026ef None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-410fb836-4d15-4bcf-b72b-ec8c7d1026ef None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:33.254994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:33.259608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:33.260735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 582/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:50:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:33.262230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:33.365203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ffff1-6dfd-4c24-be7b-6f723c9ae411 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:33.389866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:33.403923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32f3ad-1b07-4e2b-93cd-3b057ab0f8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:33.404588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f3ad-1b07-4e2b-93cd-3b057ab0f8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:33.404875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f3ad-1b07-4e2b-93cd-3b057ab0f8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:33.405136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f3ad-1b07-4e2b-93cd-3b057ab0f8f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:33.405782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32f3ad-1b07-4e2b-93cd-3b057ab0f8f8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:33.406084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32f3ad-1b07-4e2b-93cd-3b057ab0f8f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:33.407498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32f3ad-1b07-4e2b-93cd-3b057ab0f8f8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:33.414909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 584/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Sep 5 16:50:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:35.328025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56f08-9511-4490-bcd6-13ae3edd1565 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:35.328993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56f08-9511-4490-bcd6-13ae3edd1565 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:35.328993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b56f08-9511-4490-bcd6-13ae3edd1565 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:35.331175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 583/1167] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:35.336590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a40ea-6fb5-42bc-a7bd-01404e747dfa None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:35.337029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a40ea-6fb5-42bc-a7bd-01404e747dfa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:35.337135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a40ea-6fb5-42bc-a7bd-01404e747dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:35.365429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b81a40ea-6fb5-42bc-a7bd-01404e747dfa None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:35.367353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b81a40ea-6fb5-42bc-a7bd-01404e747dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:35.441839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 585/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:35.705557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56f08-9511-4490-bcd6-13ae3edd1565 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:35.822329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b56f08-9511-4490-bcd6-13ae3edd1565 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:35.857338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b34c0-04cc-443c-97a9-340d63db25dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:35.857926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b34c0-04cc-443c-97a9-340d63db25dc None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:35.858200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b34c0-04cc-443c-97a9-340d63db25dc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:35.858475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b34c0-04cc-443c-97a9-340d63db25dc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:35.859128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b34c0-04cc-443c-97a9-340d63db25dc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:35.860548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b34c0-04cc-443c-97a9-340d63db25dc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:35.879938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 584/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Mon Sep 5 16:50:35 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:50:37.853546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388b81-99f2-4e49-8fa7-4c0ea143530c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:37.854092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388b81-99f2-4e49-8fa7-4c0ea143530c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:37.854092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388b81-99f2-4e49-8fa7-4c0ea143530c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:37.855762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 586/1170] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:37.862253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e64c51-5ba2-4c82-b817-4ce82b9a0403 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:37.863032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e64c51-5ba2-4c82-b817-4ce82b9a0403 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:37.863161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e64c51-5ba2-4c82-b817-4ce82b9a0403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:37.893146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-25e64c51-5ba2-4c82-b817-4ce82b9a0403 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:37.894352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-25e64c51-5ba2-4c82-b817-4ce82b9a0403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:37.908670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:37.967438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 585/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:38.240297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b388b81-99f2-4e49-8fa7-4c0ea143530c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:38.350426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b388b81-99f2-4e49-8fa7-4c0ea143530c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:38.367300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:38.386931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833d710-0aaa-462d-989b-6ee6305c3cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:38.387564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d710-0aaa-462d-989b-6ee6305c3cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:38.387840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d710-0aaa-462d-989b-6ee6305c3cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:38.388109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d710-0aaa-462d-989b-6ee6305c3cd3 None admin] PATH_INFO: `/v3/domains/7376680dce0a4cb7a971d569373ad08c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:38.398747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833d710-0aaa-462d-989b-6ee6305c3cd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7376680dce0a4cb7a971d569373ad08c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:38.400189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833d710-0aaa-462d-989b-6ee6305c3cd3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:38.401837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 587/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Mon Sep 5 16:50:38 2022] GET /identity/v3/domains/7376680dce0a4cb7a971d569373ad08c => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:38.408085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e64c51-5ba2-4c82-b817-4ce82b9a0403 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:38.514065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e64c51-5ba2-4c82-b817-4ce82b9a0403 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:38.549254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84742997-b62d-44e9-b9af-cae3a1d0a6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:38.549852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742997-b62d-44e9-b9af-cae3a1d0a6ee None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:38.550127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742997-b62d-44e9-b9af-cae3a1d0a6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:38.550398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84742997-b62d-44e9-b9af-cae3a1d0a6ee None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:38.551041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84742997-b62d-44e9-b9af-cae3a1d0a6ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:38.552502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84742997-b62d-44e9-b9af-cae3a1d0a6ee None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:38.579273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:38.586818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 586/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:50:38 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:50:40.499411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb8ad8-4e34-45cc-b7e8-3aa74ae1905f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:40.500372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb8ad8-4e34-45cc-b7e8-3aa74ae1905f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:40.500372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb8ad8-4e34-45cc-b7e8-3aa74ae1905f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:40.502580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 588/1174] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:40.508103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecfb12-4d3d-43c9-ae25-e6c0fde28b1d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:40.508574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecfb12-4d3d-43c9-ae25-e6c0fde28b1d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:40.508673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecfb12-4d3d-43c9-ae25-e6c0fde28b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:40.533752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3fecfb12-4d3d-43c9-ae25-e6c0fde28b1d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:40.534930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3fecfb12-4d3d-43c9-ae25-e6c0fde28b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:40.607578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 587/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:40.884594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb8ad8-4e34-45cc-b7e8-3aa74ae1905f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:40.992595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb8ad8-4e34-45cc-b7e8-3aa74ae1905f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:41.033986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edbbae-af3e-4e58-ae72-c844a11c7b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:41.034621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edbbae-af3e-4e58-ae72-c844a11c7b22 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:41.034909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edbbae-af3e-4e58-ae72-c844a11c7b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:41.035199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edbbae-af3e-4e58-ae72-c844a11c7b22 None admin] PATH_INFO: `/v3/domains/7376680dce0a4cb7a971d569373ad08c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:41.036396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edbbae-af3e-4e58-ae72-c844a11c7b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7376680dce0a4cb7a971d569373ad08c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:41.037842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5edbbae-af3e-4e58-ae72-c844a11c7b22 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:41.039434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 589/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Mon Sep 5 16:50:40 2022] GET /identity/v3/domains/7376680dce0a4cb7a971d569373ad08c => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:41.046323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecfb12-4d3d-43c9-ae25-e6c0fde28b1d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:41.153963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecfb12-4d3d-43c9-ae25-e6c0fde28b1d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:41.187278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:41.197279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8f940-ae5c-4218-b6b0-fc920f28d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:41.198156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8f940-ae5c-4218-b6b0-fc920f28d0a9 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:41.198501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8f940-ae5c-4218-b6b0-fc920f28d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:41.198856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8f940-ae5c-4218-b6b0-fc920f28d0a9 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:41.199828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8f940-ae5c-4218-b6b0-fc920f28d0a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:41.202134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8f940-ae5c-4218-b6b0-fc920f28d0a9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:41.232467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 588/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:50:41 2022] POST /identity/v3/users => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:50:43.165333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb58e2c-1f45-483a-8ba0-10bc82ae5206 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:43.166262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb58e2c-1f45-483a-8ba0-10bc82ae5206 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:43.166262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb58e2c-1f45-483a-8ba0-10bc82ae5206 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:43.168446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 590/1178] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:43.174175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3788804-87a3-4f62-9926-580bb4768cfb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:43.174616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3788804-87a3-4f62-9926-580bb4768cfb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:43.174717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3788804-87a3-4f62-9926-580bb4768cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:43.194310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c3788804-87a3-4f62-9926-580bb4768cfb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:43.195494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c3788804-87a3-4f62-9926-580bb4768cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:43.265451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 589/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:43.531381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb58e2c-1f45-483a-8ba0-10bc82ae5206 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:43.584436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:43.644991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb58e2c-1f45-483a-8ba0-10bc82ae5206 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:43.680025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:43.680682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:43.680960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:43.681230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:43.692301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:43.697127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:43.698307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:50:43.699317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 591/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:50:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:43.704557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3788804-87a3-4f62-9926-580bb4768cfb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:43.806650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3788804-87a3-4f62-9926-580bb4768cfb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:43.823157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:43.849566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:43.850431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:43.850818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:43.851187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:43.852221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:43.852636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:43.854796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:43.865608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 590/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:43.871422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:43.976823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b520e-7c27-4ecf-b2c5-75aeddfe0aee None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:43.979361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:44.014101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:44.014729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:44.015032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:44.015282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] PATH_INFO: `/v3/users/f5e35e994d1f4715a6a8716ddb39652c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:44.029754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] RBAC: Authorizing `identity:get_user(user_id=f5e35e994d1f4715a6a8716ddb39652c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:44.031198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:44.032903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 592/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:43 2022] GET /identity/v3/users/f5e35e994d1f4715a6a8716ddb39652c => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:44.039594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:44.146031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5401311-4271-4fac-a409-5fcc4f60f73f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:44.181081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:44.181692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:44.181938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:44.182226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:44.229511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:44.252191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:44.253651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:44.351528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 591/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:50:44.361583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:44.473439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023c323-b0e0-408f-ab13-a32e504e5213 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:44.517130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019fd1a-83f8-4eb4-8688-2a943b6fd4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:44.517777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019fd1a-83f8-4eb4-8688-2a943b6fd4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:44.518076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019fd1a-83f8-4eb4-8688-2a943b6fd4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:44.518347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019fd1a-83f8-4eb4-8688-2a943b6fd4f4 None admin] PATH_INFO: `/v3/projects/76afde82ad4046cc9ae03d842ad723d7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:44.519495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019fd1a-83f8-4eb4-8688-2a943b6fd4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=76afde82ad4046cc9ae03d842ad723d7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:44.520955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019fd1a-83f8-4eb4-8688-2a943b6fd4f4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:44.522572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 593/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:44 2022] GET /identity/v3/projects/76afde82ad4046cc9ae03d842ad723d7 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:44.529052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:44.610347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:44.649264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733207a-d903-4797-89b5-d4f5a8a17bf3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:44.689173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff9aaa-f4e2-4e66-bbca-1023ec6a6cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:44.689777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9aaa-f4e2-4e66-bbca-1023ec6a6cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:44.690060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9aaa-f4e2-4e66-bbca-1023ec6a6cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:44.690336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff9aaa-f4e2-4e66-bbca-1023ec6a6cd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:44.691074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff9aaa-f4e2-4e66-bbca-1023ec6a6cd3 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=76afde82ad4046cc9ae03d842ad723d7, user.id=f5e35e994d1f4715a6a8716ddb39652c) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:44.691379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff9aaa-f4e2-4e66-bbca-1023ec6a6cd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:44.692841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff9aaa-f4e2-4e66-bbca-1023ec6a6cd3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:44.698352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 592/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:44 2022] GET /identity/v3/role_assignments?user.id=f5e35e994d1f4715a6a8716ddb39652c&scope.project.id=76afde82ad4046cc9ae03d842ad723d7&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:46.555500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8c853-f2b0-4d17-a4e9-1d4b7ff4928e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:46.556087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8c853-f2b0-4d17-a4e9-1d4b7ff4928e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:46.556087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8c853-f2b0-4d17-a4e9-1d4b7ff4928e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:46.557421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 594/1186] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:46.562960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a690d1-76db-40f0-ae1a-40f6b6365360 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:46.563429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a690d1-76db-40f0-ae1a-40f6b6365360 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:46.563792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a690d1-76db-40f0-ae1a-40f6b6365360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:46.590637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-18a690d1-76db-40f0-ae1a-40f6b6365360 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:46.592599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-18a690d1-76db-40f0-ae1a-40f6b6365360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:46.666955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 593/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:46.940530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8c853-f2b0-4d17-a4e9-1d4b7ff4928e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:46.976442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:47.051581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8c853-f2b0-4d17-a4e9-1d4b7ff4928e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:47.086878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:47.087501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:47.087778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:47.088053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:47.098947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:47.103744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:47.104857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:50:47.106017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 595/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:47.113485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a690d1-76db-40f0-ae1a-40f6b6365360 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:47.191146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:47.222725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a690d1-76db-40f0-ae1a-40f6b6365360 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:47.259251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:47.260114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:47.260540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:47.260905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:47.261914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:47.262312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:47.264500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:47.279549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 594/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:47.286683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:47.389740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:47.401162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615c0b4-4c6c-48e3-b32d-48a5100e9d53 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:47.435787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:47.436444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:47.436729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:47.437001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] PATH_INFO: `/v3/users/f5e35e994d1f4715a6a8716ddb39652c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:47.438221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] RBAC: Authorizing `identity:get_user(user_id=f5e35e994d1f4715a6a8716ddb39652c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:47.439699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:47.441446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 596/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:47 2022] GET /identity/v3/users/f5e35e994d1f4715a6a8716ddb39652c => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:47.447537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:47.556308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5eae9-a2d0-4128-82cc-827dc6b627fb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:47.591389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337845ef-2fd6-4389-b5c7-7d9c04ca2b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:47.592010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337845ef-2fd6-4389-b5c7-7d9c04ca2b84 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:47.592312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337845ef-2fd6-4389-b5c7-7d9c04ca2b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:47.592583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337845ef-2fd6-4389-b5c7-7d9c04ca2b84 None admin] PATH_INFO: `/v3/projects/76afde82ad4046cc9ae03d842ad723d7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:47.593730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337845ef-2fd6-4389-b5c7-7d9c04ca2b84 None admin] RBAC: Authorizing `identity:get_project(project_id=76afde82ad4046cc9ae03d842ad723d7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:47.595324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337845ef-2fd6-4389-b5c7-7d9c04ca2b84 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:47.597014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 595/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:47 2022] GET /identity/v3/projects/76afde82ad4046cc9ae03d842ad723d7 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:50:47.606895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:47.718703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499a6d8-9a7c-4eb2-b357-8a7fcc608446 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:47.756704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca00fb-f77c-4b0b-9436-0a6658b98002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:47.757360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca00fb-f77c-4b0b-9436-0a6658b98002 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:47.757641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca00fb-f77c-4b0b-9436-0a6658b98002 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:47.757916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca00fb-f77c-4b0b-9436-0a6658b98002 None admin] PATH_INFO: `/v3/projects/76afde82ad4046cc9ae03d842ad723d7/users/f5e35e994d1f4715a6a8716ddb39652c/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:47.760099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca00fb-f77c-4b0b-9436-0a6658b98002 None admin] RBAC: Authorizing `identity:create_grant(project_id=76afde82ad4046cc9ae03d842ad723d7, user_id=f5e35e994d1f4715a6a8716ddb39652c, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:47.761579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ca00fb-f77c-4b0b-9436-0a6658b98002 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:47.781818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 597/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:50:47 2022] PUT /identity/v3/projects/76afde82ad4046cc9ae03d842ad723d7/users/f5e35e994d1f4715a6a8716ddb39652c/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:50:49.673802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cd5f-ac8a-47fc-8ba3-61938c1b3191 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:49.675877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cd5f-ac8a-47fc-8ba3-61938c1b3191 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:49.676415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82cd5f-ac8a-47fc-8ba3-61938c1b3191 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:49.678611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 596/1193] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:50:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:50:49.684917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e26886-dfe2-451f-8e95-1749fafbc716 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:49.685683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e26886-dfe2-451f-8e95-1749fafbc716 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:49.685833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e26886-dfe2-451f-8e95-1749fafbc716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:49.705946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d6e26886-dfe2-451f-8e95-1749fafbc716 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:50:49.707147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d6e26886-dfe2-451f-8e95-1749fafbc716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:50:49.721541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:49.790530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 598/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:50:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:50:50.059813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82cd5f-ac8a-47fc-8ba3-61938c1b3191 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:50.064449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:50.175698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82cd5f-ac8a-47fc-8ba3-61938c1b3191 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:50.212986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:50.213594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:50.213862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:50.214126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:50.225049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:50:50.229757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:50:50.231125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:50:50.232195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 597/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:50:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:50:50.237596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e26886-dfe2-451f-8e95-1749fafbc716 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:50.344642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e26886-dfe2-451f-8e95-1749fafbc716 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:50.379975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:50.380617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:50.380907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:50.381175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:50.381844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:50.382145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:50.383530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:50.392090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 599/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:50:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:50.398136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:50.474332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:50.525014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f797cd34-9b4d-438d-9dd4-7aa1d4ae812b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:50.562034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:50.562657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:50.562926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:50.563191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] PATH_INFO: `/v3/users/f5e35e994d1f4715a6a8716ddb39652c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:50.564363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] RBAC: Authorizing `identity:get_user(user_id=f5e35e994d1f4715a6a8716ddb39652c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:50.565787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:50.567367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 598/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Sep 5 16:50:50 2022] GET /identity/v3/users/f5e35e994d1f4715a6a8716ddb39652c => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:50.577025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:50.591160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:50.683457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a7397-8f00-48c9-9932-354d8b738477 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:50.722458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:50.723100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:50.723335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:50.723612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:50.794086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:50.795541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:50.806728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:50.899135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 600/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:50:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:50:50.905578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:51.014432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d17c40-6136-4691-97d5-cf08f80eee89 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:51.044708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:51.052560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503ccbb-7318-4e94-b601-b769dbe722d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:51.053179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503ccbb-7318-4e94-b601-b769dbe722d3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:51.053456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503ccbb-7318-4e94-b601-b769dbe722d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:51.053719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503ccbb-7318-4e94-b601-b769dbe722d3 None admin] PATH_INFO: `/v3/projects/76afde82ad4046cc9ae03d842ad723d7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:51.054895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503ccbb-7318-4e94-b601-b769dbe722d3 None admin] RBAC: Authorizing `identity:get_project(project_id=76afde82ad4046cc9ae03d842ad723d7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:51.056346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503ccbb-7318-4e94-b601-b769dbe722d3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:51.057946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 599/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Sep 5 16:50:50 2022] GET /identity/v3/projects/76afde82ad4046cc9ae03d842ad723d7 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:50:51.066472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:50:51.167251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cfe78-5f13-4bfb-96a6-069b5ad32aa3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:50:51.178279 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:50:51.209736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c06c5-3df2-49f3-8ba9-48c9201f0165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:50:51.210352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c06c5-3df2-49f3-8ba9-48c9201f0165 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:50:51.210643 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c06c5-3df2-49f3-8ba9-48c9201f0165 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:50:51.210916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c06c5-3df2-49f3-8ba9-48c9201f0165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:50:51.211640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c06c5-3df2-49f3-8ba9-48c9201f0165 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=76afde82ad4046cc9ae03d842ad723d7, user.id=f5e35e994d1f4715a6a8716ddb39652c) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:50:51.211946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c06c5-3df2-49f3-8ba9-48c9201f0165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:50:51.213445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c06c5-3df2-49f3-8ba9-48c9201f0165 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:50:51.220010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 601/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:50:51 2022] GET /identity/v3/role_assignments?user.id=f5e35e994d1f4715a6a8716ddb39652c&scope.project.id=76afde82ad4046cc9ae03d842ad723d7&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:01.051302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:01.216646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=13 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:46.358590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a445-783b-4aaf-bf5e-f3175c0dce86 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:46.359333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a445-783b-4aaf-bf5e-f3175c0dce86 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:46.359333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a445-783b-4aaf-bf5e-f3175c0dce86 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:46.360828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 600/1201] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:51:46.366371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32cae-a7bd-4ae1-b61f-97a19014d21c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:46.366899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32cae-a7bd-4ae1-b61f-97a19014d21c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:46.366899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32cae-a7bd-4ae1-b61f-97a19014d21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:46.386352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d4f32cae-a7bd-4ae1-b61f-97a19014d21c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:51:46.387556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d4f32cae-a7bd-4ae1-b61f-97a19014d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:51:46.475604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 602/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:51:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:51:46.747140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a445-783b-4aaf-bf5e-f3175c0dce86 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:46.872821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a445-783b-4aaf-bf5e-f3175c0dce86 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:46.915243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:46.915892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:46.916252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:46.916635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:46.929960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:46.931633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:51:46.949187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:46.950289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:51:46.951329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:51:46.951329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:46.951329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:51:46.951329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 601/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:46.958556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32cae-a7bd-4ae1-b61f-97a19014d21c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:47.108231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32cae-a7bd-4ae1-b61f-97a19014d21c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:47.143811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:47.151343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8158568-907e-4f69-9019-dd767497def0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:47.152041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158568-907e-4f69-9019-dd767497def0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:47.152410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158568-907e-4f69-9019-dd767497def0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:47.152644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158568-907e-4f69-9019-dd767497def0 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:47.153313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8158568-907e-4f69-9019-dd767497def0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:47.153605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8158568-907e-4f69-9019-dd767497def0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:47.154981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8158568-907e-4f69-9019-dd767497def0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:47.173659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 603/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:47.179624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:47.245349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:47.305914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741020a3-f59c-4c93-b8d3-449af5fb866e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:47.348590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308e444-1c32-489f-b247-0affa0dc9366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:47.349255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308e444-1c32-489f-b247-0affa0dc9366 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:47.349508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308e444-1c32-489f-b247-0affa0dc9366 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:47.349797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308e444-1c32-489f-b247-0affa0dc9366 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:47.350592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308e444-1c32-489f-b247-0affa0dc9366 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:47.352077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308e444-1c32-489f-b247-0affa0dc9366 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:47.384186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 602/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Sep 5 16:51:47 2022] POST /identity/v3/users => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:51:49.476193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd863-ce64-483a-9160-1b5e6794b5d2 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:49.476825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd863-ce64-483a-9160-1b5e6794b5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:49.476825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fd863-ce64-483a-9160-1b5e6794b5d2 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:49.478164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 604/1206] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:51:49.484427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f31c1-03f4-4415-b927-e13518f0a7c8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:49.484903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f31c1-03f4-4415-b927-e13518f0a7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:49.484995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f31c1-03f4-4415-b927-e13518f0a7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:49.504227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-372f31c1-03f4-4415-b927-e13518f0a7c8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:51:49.505494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-372f31c1-03f4-4415-b927-e13518f0a7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:51:49.581758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 603/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:51:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:51:49.859858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fd863-ce64-483a-9160-1b5e6794b5d2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:49.978951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fd863-ce64-483a-9160-1b5e6794b5d2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:50.024430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:50.025178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:50.025461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:50.025738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:50.038409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:51:50.043290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:50.044346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:50.045193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 605/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:51:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:50.051111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f31c1-03f4-4415-b927-e13518f0a7c8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:50.097833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:50.166158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f31c1-03f4-4415-b927-e13518f0a7c8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:50.202964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:50.203596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:50.203877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:50.204177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:50.204940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:50.205227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:50.206763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:50.216183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 604/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:51:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:50.221801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:50.281066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:50.347356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445652d-0f64-4742-ab29-124f29ed96b6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:50.398910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:50.399599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:50.399881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:50.400180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:50.413875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:50.415431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:51:50.430877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:50.431909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:51:50.433009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:51:50.433009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:50.433009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:50.433009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 606/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:50.443428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:50.556022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:50.559491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071e5e6-ba3d-4b25-83f8-c7d1c06ac043 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:50.595648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:50.596295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:50.596573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:50.596848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:50.597548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:50.597804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:50.599159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:50.616644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 605/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:50.618043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:50.725850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145ed02-e510-428f-a36b-2ab8e2f6ba5e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:50.727396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:50.763305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:50.764116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:50.764492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:50.764924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:50.773983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:50.775529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:51:50.787160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:50.793682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:50.794657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 607/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:50.795296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:50.900703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a97087-f62c-476d-a037-b75817aad87e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:50.938981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:50.939599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:50.939884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:50.940172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:50.940931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:50.941213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:50.942637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:50.964916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 606/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Sep 5 16:51:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:50.966133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:51.081689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a310d28-e805-4e8c-9d3b-195eb4e26bb7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:51.122070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:51.122698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:51.122982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:51.123318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:51.157584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:51.203551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:51.205052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:51.320904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 608/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:51:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 05 16:51:51.327101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:51.349378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:51.441168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc44a-386f-4fea-bca3-3cf7fb3820a7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:51.478315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:51.479059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:51.479419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:51.479758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:51.491241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:51.492802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:51:51.508404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:51.509405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:51:51.510304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:51:51.510304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:51.510304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:51:51.510304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 607/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:51.515372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:51.588826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:51.639078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93431603-a886-4228-90dd-e6620444f8c6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:51.677246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:51.677865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:51.678140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:51.678407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:51.679043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:51.679341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:51.680831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:51.692765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 609/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:51.698316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:51.778509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:51.814462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ce70b-bf71-493c-9784-44c84cb57709 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:51.851188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:51.852055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:51.852464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:51.852828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:51.861096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:51.862917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:51:51.874973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:51.876190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:51:51.877343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:51:51.878714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 608/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:51:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:51.885053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:51.993417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012065fe-dda4-4fa3-a76d-334097cbf144 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:52.017718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:52.030305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28e3f2-f037-4acf-8064-215e1f3df2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:52.031113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e3f2-f037-4acf-8064-215e1f3df2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:52.031530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e3f2-f037-4acf-8064-215e1f3df2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:52.031919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28e3f2-f037-4acf-8064-215e1f3df2d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:52.032729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28e3f2-f037-4acf-8064-215e1f3df2d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:52.033128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28e3f2-f037-4acf-8064-215e1f3df2d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:52.034651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28e3f2-f037-4acf-8064-215e1f3df2d0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:52.048974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 610/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:51:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:52.055002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:52.172178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd31049-eb87-49da-99a5-60ebd0d32613 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:52.212092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:52.219010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86303cd6-0692-4cfb-ad72-72e477c0cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:52.219620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303cd6-0692-4cfb-ad72-72e477c0cad4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:52.219956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303cd6-0692-4cfb-ad72-72e477c0cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:52.220185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303cd6-0692-4cfb-ad72-72e477c0cad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:52.220919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303cd6-0692-4cfb-ad72-72e477c0cad4 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2b0d695705124ac0b5d0a63e70ad45ce) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:52.221240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303cd6-0692-4cfb-ad72-72e477c0cad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:52.222650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303cd6-0692-4cfb-ad72-72e477c0cad4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:52.230446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 609/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:51:52 2022] GET /identity/v3/role_assignments?user.id=2b0d695705124ac0b5d0a63e70ad45ce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:51:54.200492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f8a2-0446-4a8e-8843-caf3372f1798 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:54.201069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f8a2-0446-4a8e-8843-caf3372f1798 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:54.201175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f8a2-0446-4a8e-8843-caf3372f1798 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:54.202810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 611/1220] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:51:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:51:54.208416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc837196-4d32-4832-afd0-a3c62eb8ef1c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:54.208896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc837196-4d32-4832-afd0-a3c62eb8ef1c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:54.208989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc837196-4d32-4832-afd0-a3c62eb8ef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:54.230178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cc837196-4d32-4832-afd0-a3c62eb8ef1c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:51:54.231538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cc837196-4d32-4832-afd0-a3c62eb8ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:51:54.305990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 610/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:51:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:51:54.578562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948f8a2-0446-4a8e-8843-caf3372f1798 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:54.697948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948f8a2-0446-4a8e-8843-caf3372f1798 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:54.732291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:54.732939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:54.733275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:54.733633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:54.745727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:51:54.752706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:54.753745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:54.754780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 612/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:51:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:54.760276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc837196-4d32-4832-afd0-a3c62eb8ef1c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:54.871928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc837196-4d32-4832-afd0-a3c62eb8ef1c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:54.897519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:54.908421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:54.909025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:54.909297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:54.909606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:54.910257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:54.910555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:54.911912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:54.920251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 611/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:51:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:54.926886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:54.975584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:55.033471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f5902-ab33-4a7b-a467-f3d40fa62a41 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:55.069171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:55.070018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:55.070295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:55.070562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:55.089763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:55.091238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:51:55.107066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:55.108235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:51:55.109446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:51:55.109446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:55.109446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:55.112471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 613/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:55.114390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:55.221166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38453e49-b7f4-4d34-96aa-bc920172601d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:55.255501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:55.256187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:55.256487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:55.256785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:55.257491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:55.257814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:55.259141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:55.271300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 612/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:55.277219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:55.376477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:55.380202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856e5d5-1855-471e-8d29-6635f928ccc2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:55.414914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:55.415552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:55.415828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:55.416097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:55.423907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:55.425384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:51:55.436914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:55.438133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:55.439305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 614/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:55.444128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:55.495657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:55.549302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab59a4dc-b354-4803-808b-60a57b3353e4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:55.584132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:55.584792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:55.585068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:55.585346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:55.586128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:55.586446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:55.587850 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:55.605774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 613/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Sep 5 16:51:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:55.611964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:55.720931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576bc34-d57d-44ae-9938-d5a9e7dfbcc6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:55.759892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:55.760922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:55.761396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:55.761832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:55.777213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:55.779220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:51:55.800612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:55.801851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:51:55.802954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:51:55.802954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:55.802954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:55.802954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 615/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:55.809681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:55.930284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cca38c-72eb-4c12-9e79-75cc4795ab31 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:55.933379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:55.972675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:55.973385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:55.973665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:55.973939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:55.974577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:55.974880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:55.976312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:55.989142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 614/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:51:55.996322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:56.007938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:56.105901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f3502-b8ea-4e90-ac31-e96031469277 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:56.141472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:56.142211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:56.142503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:56.142774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:56.149736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:56.151263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:51:56.162726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:56.163821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:51:56.164986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 616/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:56.170051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:56.278876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0902c2-c812-477d-8f76-ef1f8199b82b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:56.314056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f52d9-da42-42db-8a8e-75940cb14565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:56.314611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f52d9-da42-42db-8a8e-75940cb14565 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:56.314893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f52d9-da42-42db-8a8e-75940cb14565 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:56.315155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f52d9-da42-42db-8a8e-75940cb14565 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:56.315831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f52d9-da42-42db-8a8e-75940cb14565 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:56.316134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f52d9-da42-42db-8a8e-75940cb14565 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:56.317620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f52d9-da42-42db-8a8e-75940cb14565 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:56.329687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 615/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:51:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:56.336104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:56.409811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:56.446441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba463e3-da28-4205-9549-5c1ab2e4beca None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:56.481626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0b48d-8745-4cc6-a2d1-27cce78718d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:56.482280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0b48d-8745-4cc6-a2d1-27cce78718d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:56.482561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0b48d-8745-4cc6-a2d1-27cce78718d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:56.482865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0b48d-8745-4cc6-a2d1-27cce78718d1 None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2b0d695705124ac0b5d0a63e70ad45ce/roles/ba985e88b6d44ba38445fe86872360b2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:56.501657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0b48d-8745-4cc6-a2d1-27cce78718d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=2b0d695705124ac0b5d0a63e70ad45ce, role_id=ba985e88b6d44ba38445fe86872360b2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:56.503298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0b48d-8745-4cc6-a2d1-27cce78718d1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:56.518327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 617/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:51:56 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2b0d695705124ac0b5d0a63e70ad45ce/roles/ba985e88b6d44ba38445fe86872360b2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:51:58.618057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56c697-fdf0-46f2-862f-a00148646ef2 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:58.618684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56c697-fdf0-46f2-862f-a00148646ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:58.618684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f56c697-fdf0-46f2-862f-a00148646ef2 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:58.620023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 616/1233] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:51:58.626198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65329f-9966-4240-8c73-5532328db54c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:58.626628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65329f-9966-4240-8c73-5532328db54c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:58.626705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65329f-9966-4240-8c73-5532328db54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:58.645882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-db65329f-9966-4240-8c73-5532328db54c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:51:58.647077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-db65329f-9966-4240-8c73-5532328db54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:51:58.718763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 618/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:51:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:51:58.986758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56c697-fdf0-46f2-862f-a00148646ef2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:59.012054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:59.095462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f56c697-fdf0-46f2-862f-a00148646ef2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:59.132066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:59.132740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:59.133018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:59.133288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:59.145335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:51:59.150390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:59.151623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:51:59.152566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 617/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Sep 5 16:51:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:59.158406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65329f-9966-4240-8c73-5532328db54c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:59.213454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:59.276898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65329f-9966-4240-8c73-5532328db54c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:59.313225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:59.313842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:59.314118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:59.314386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:59.315112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:59.315410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:59.316846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:59.325168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 619/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Sep 5 16:51:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:51:59.331274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:59.419068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:59.436625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb6012d-3d77-4e8f-8732-4f02e2b6c706 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:59.472401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:59.474374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:59.474374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:59.474374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:59.485065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:59.486520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:51:59.501734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:59.503033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:51:59.503935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:51:59.503935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:51:59.503935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:51:59.503935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 618/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:51:59.509088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:59.616335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc656a-0d7b-4a0a-98af-92ba0554e19f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:59.626899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:59.653984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:59.654647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:59.654951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:59.655303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:59.656005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:51:59.656345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:59.657759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:59.670840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 620/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:51:59.676584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:59.783830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071c120-45a8-437d-a656-ff0eff79bcdf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:51:59.819549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:51:59.820214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:51:59.820493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:51:59.820815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:51:59.823727 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:51:59.830115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:51:59.831824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:51:59.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:51:59.843928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:51:59.844994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 619/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:51:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:51:59.850395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:51:59.967826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bd62b-4f8d-4d03-a285-84b39ce76bcd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:00.004782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:00.005402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:00.005682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:00.005956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:00.006682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:00.006989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:00.008472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:00.027518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 621/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Sep 5 16:51:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:00.034342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:00.146881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feff4d7-d348-4251-b249-f7c793c09a64 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:00.186615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:00.187229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:00.187619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:00.187889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:00.258088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:00.265664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:00.267131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:00.378052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 620/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:52:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:52:00.383945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:00.412221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:00.491370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390074dd-6d34-4ece-801a-2e62d211ebec None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:00.531344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:00.531915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:00.532215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:00.532556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:00.543929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:00.545576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:52:00.561342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:00.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:52:00.563361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:52:00.563361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:00.563361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:00.563361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 622/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:00.568581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:00.680778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:00.688015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1168660-0b7e-4519-94db-63638a453e37 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:00.727581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:00.728220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:00.728533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:00.728843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:00.729673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:00.730369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:00.732035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:00.746939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 621/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:00.752569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:00.845296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:00.868658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed3d9f-3875-4930-b567-5a4d34bd2f33 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:00.904381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:00.905084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:00.905360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:00.905634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:00.912698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:00.914154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:52:00.926108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:00.927230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:00.933163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 623/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:52:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 05 16:52:00.933766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:01.044975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee250f1-6e28-4974-8ef2-21a9c4518c44 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:01.081728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63c4ad-f119-46fd-9ff5-3a2b926b51f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:01.082331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c4ad-f119-46fd-9ff5-3a2b926b51f4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:01.082602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c4ad-f119-46fd-9ff5-3a2b926b51f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:01.082869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63c4ad-f119-46fd-9ff5-3a2b926b51f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:01.083543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63c4ad-f119-46fd-9ff5-3a2b926b51f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:01.083842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63c4ad-f119-46fd-9ff5-3a2b926b51f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:01.085296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63c4ad-f119-46fd-9ff5-3a2b926b51f4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:01.098546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 622/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:52:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:52:01.104716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:01.213452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da2fe-51b4-4b41-bc9a-7ff07756baf1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:01.248868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f1621-513b-45ca-8986-0533e2656924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:01.249421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f1621-513b-45ca-8986-0533e2656924 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:01.249701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f1621-513b-45ca-8986-0533e2656924 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:01.249978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f1621-513b-45ca-8986-0533e2656924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:01.250696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f1621-513b-45ca-8986-0533e2656924 None admin] RBAC: Adding query filter params (role.id=ba985e88b6d44ba38445fe86872360b2, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2b0d695705124ac0b5d0a63e70ad45ce) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:01.251002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f1621-513b-45ca-8986-0533e2656924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:01.252511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f1621-513b-45ca-8986-0533e2656924 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:01.258315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 624/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:52:01 2022] GET /identity/v3/role_assignments?user.id=2b0d695705124ac0b5d0a63e70ad45ce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=ba985e88b6d44ba38445fe86872360b2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:03.333517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68aaaa-7767-4f32-9055-28d66357af1c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:03.334056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68aaaa-7767-4f32-9055-28d66357af1c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:03.334056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68aaaa-7767-4f32-9055-28d66357af1c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:03.335432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 623/1247] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:52:03.340919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017efe94-f589-447d-b7ba-a6ef070501be None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:03.341388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017efe94-f589-447d-b7ba-a6ef070501be None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:03.341489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017efe94-f589-447d-b7ba-a6ef070501be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:03.348356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:03.363581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-017efe94-f589-447d-b7ba-a6ef070501be None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:52:03.364846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-017efe94-f589-447d-b7ba-a6ef070501be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:52:03.437701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 625/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:52:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 05 16:52:03.702112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68aaaa-7767-4f32-9055-28d66357af1c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:03.721622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:03.819962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68aaaa-7767-4f32-9055-28d66357af1c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:03.857622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:03.858523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:03.858804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:03.859073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:03.869809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:52:03.874804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:03.875923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:03.876961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 624/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:52:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:03.882427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017efe94-f589-447d-b7ba-a6ef070501be None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:03.999117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017efe94-f589-447d-b7ba-a6ef070501be None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:04.034812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:04.041025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:04.041678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:04.041960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:04.042230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:04.042886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:04.043186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:04.044755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:04.054204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 626/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:52:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:04.060263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:04.153455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:04.180194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be083f99-94dd-483d-a4e0-07b9babe12c9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:04.221141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:04.221821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:04.222026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:04.222315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:04.234504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:04.235964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:52:04.251533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:04.253547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:52:04.255636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:52:04.255636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:04.255636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:04.255636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 625/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:04.259144 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:04.373557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab85a4d-22ed-4c84-90b1-24366f1ec05b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:04.412877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:04.413519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:04.413974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:04.414274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:04.415026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:04.415486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:04.416921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:04.430509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 627/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:04.436514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:04.558515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454280b8-7a05-4dab-b0d2-2ab418f49f0e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:04.588947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:04.600451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:04.601161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:04.601462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:04.601735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:04.611007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:04.612585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:52:04.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:04.627418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:52:04.628971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:04.632316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 626/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:04.633304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:04.703273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:04.752899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877bf7d-5db0-4f1c-a5dc-ef1fd0397e83 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:04.788846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:04.789522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:04.789849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:04.790160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:04.792757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:04.792757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:04.792757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:04.812581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 628/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Sep 5 16:52:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:52:04.818561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:04.938529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e6776-4e4f-40a0-bf4f-1532a973935a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:04.982321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:04.982934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:04.983276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:04.983564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:05.042985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:05.057250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:05.058815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:05.173062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 627/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:52:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:52:05.174402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:05.287659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a865ef-b20e-45ee-93b7-b8e3013a1c8a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:05.305267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:05.327703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:05.328442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:05.328756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:05.329031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:05.340563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:05.342333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:52:05.357537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:05.359301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:52:05.360253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:52:05.360253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:05.360253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:05.360253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 629/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:05.364781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:05.462390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:05.474627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5be27-03f8-4c88-a7d6-729147b4f893 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:05.510144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:05.510755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:05.511024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:05.511327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:05.511965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:05.512336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:05.513772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:05.525994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 628/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:05.532579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:05.641248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ccc4e-f1fc-4472-9740-20471ea6f606 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:05.675884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:05.676618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:05.676931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:05.677224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:05.684543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:05.686161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:52:05.699852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:05.700947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:05.702249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:05.706652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 630/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:05.707735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:05.814493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9093bd0-b761-4258-b864-273b69e0d1dd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:05.849845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f2f8f1-d98f-4976-baeb-1dc999e64cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:05.850453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2f8f1-d98f-4976-baeb-1dc999e64cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:05.850727 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2f8f1-d98f-4976-baeb-1dc999e64cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:05.850995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f2f8f1-d98f-4976-baeb-1dc999e64cf6 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:05.851675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2f8f1-d98f-4976-baeb-1dc999e64cf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:05.851977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2f8f1-d98f-4976-baeb-1dc999e64cf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:05.853440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f2f8f1-d98f-4976-baeb-1dc999e64cf6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:05.866290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 629/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:05.872855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:05.901613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:06.002982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3ac01-d2ec-4798-bbb4-78488f475456 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:06.039028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d40c5e-dba7-48cb-9f6a-0d98e099b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:06.039640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40c5e-dba7-48cb-9f6a-0d98e099b13a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:06.039935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40c5e-dba7-48cb-9f6a-0d98e099b13a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:06.040233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d40c5e-dba7-48cb-9f6a-0d98e099b13a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:06.041047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d40c5e-dba7-48cb-9f6a-0d98e099b13a None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2b0d695705124ac0b5d0a63e70ad45ce) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:06.041359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d40c5e-dba7-48cb-9f6a-0d98e099b13a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:06.042803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d40c5e-dba7-48cb-9f6a-0d98e099b13a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:06.048533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 631/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:52:05 2022] GET /identity/v3/role_assignments?user.id=2b0d695705124ac0b5d0a63e70ad45ce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:07.917124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705f5f8-97c1-429e-bae0-8d0f70a3aa9a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:07.918117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705f5f8-97c1-429e-bae0-8d0f70a3aa9a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:07.918117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705f5f8-97c1-429e-bae0-8d0f70a3aa9a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:07.920016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 630/1261] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:52:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:52:07.925995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3dee-f16a-4f3b-bfd8-27a3efd4bfea None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:07.926486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3dee-f16a-4f3b-bfd8-27a3efd4bfea None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:07.926639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d3dee-f16a-4f3b-bfd8-27a3efd4bfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:07.947351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9a7d3dee-f16a-4f3b-bfd8-27a3efd4bfea None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:52:07.948620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a7d3dee-f16a-4f3b-bfd8-27a3efd4bfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:52:08.018146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 632/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:52:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:52:08.284019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705f5f8-97c1-429e-bae0-8d0f70a3aa9a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:08.296491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:08.401879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705f5f8-97c1-429e-bae0-8d0f70a3aa9a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:08.446739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:08.447356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:08.447629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:08.447893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:08.459211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:52:08.464045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:08.465112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:08.466057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 631/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:08.471589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3dee-f16a-4f3b-bfd8-27a3efd4bfea None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:08.491525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:08.586702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d3dee-f16a-4f3b-bfd8-27a3efd4bfea None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:08.621444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:08.622046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:08.622317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:08.622543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:08.623212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:08.623551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:08.625039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:08.632945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 633/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:08.641433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:08.716067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:08.751906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e180ea-1806-4f1a-be2f-3cd9658a97c5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:08.786914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:08.787525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:08.787798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:08.788026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:08.798790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:08.800261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:52:08.815948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:08.817090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:52:08.817989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:52:08.817989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:08.817989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:08.817989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 632/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:08.825040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:08.909299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:08.933247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6da929-51c7-4483-b478-6147c4dab414 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:08.970071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:08.970665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:08.970940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:08.971204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:08.971837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:08.972134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:08.973530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:08.986011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 634/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:08.991925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:09.097267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18b1d4-8105-4c70-aa76-d29055f77b89 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:09.117381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:09.133994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:09.134607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:09.134883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:09.135149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:09.143702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:09.145166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:52:09.157291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:09.158353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:09.164420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 633/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:09.165391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:09.284018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12cf86-1de0-4577-ae0a-5b3cb7e36604 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:09.323698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:09.324329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:09.324602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:09.324873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:09.325589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:09.325892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:09.327393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:09.336791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:09.346467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 635/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Sep 5 16:52:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:09.353040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:09.464842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a9437-e3c9-49c2-8c4a-b0110f1eedb2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:09.500749 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:09.501364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:09.501639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:09.501907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:09.512623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:09.514068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:52:09.528469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:09.530697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:52:09.531833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:52:09.531833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:09.531833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:09.531833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 634/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:09.535477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:09.655076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace6c1e-a1e5-4edc-956c-86f4e3d6a3b2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:09.691308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:09.691905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:09.692196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:09.692488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:09.693094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:09.693423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:09.694794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:09.706232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 636/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:09.715024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:09.749387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:09.820391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60533d46-4ac4-4238-b092-cccce0974932 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:09.856305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:09.856914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:09.857187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:09.857451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:09.864892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:09.866306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:52:09.877743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:09.878735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:09.880417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 635/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:09.887572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:09.955661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:10.001906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55472c-9524-4725-a40d-8bed1e62778c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:10.037497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea310515-6365-47a3-a029-02cc983eb6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:10.038100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea310515-6365-47a3-a029-02cc983eb6bd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:10.038378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea310515-6365-47a3-a029-02cc983eb6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:10.038655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea310515-6365-47a3-a029-02cc983eb6bd None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:10.039344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea310515-6365-47a3-a029-02cc983eb6bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:10.039649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea310515-6365-47a3-a029-02cc983eb6bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:10.041129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea310515-6365-47a3-a029-02cc983eb6bd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:10.053249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 637/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:10.059545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:10.154917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:10.170756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed2ed7e-de2b-4302-b260-d821b0ed087c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:10.204909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a0cfb-1cd3-4008-9b77-489627ae4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:10.205562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0cfb-1cd3-4008-9b77-489627ae4460 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:10.205842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0cfb-1cd3-4008-9b77-489627ae4460 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:10.206125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a0cfb-1cd3-4008-9b77-489627ae4460 None admin] PATH_INFO: `/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2b0d695705124ac0b5d0a63e70ad45ce/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:10.208281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a0cfb-1cd3-4008-9b77-489627ae4460 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bbf6ebf764fe08027e5e11c0e65c1, user_id=2b0d695705124ac0b5d0a63e70ad45ce, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:10.209712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a0cfb-1cd3-4008-9b77-489627ae4460 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:10.224004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 636/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Sep 5 16:52:10 2022] PUT /identity/v3/projects/ba6bbf6ebf764fe08027e5e11c0e65c1/users/2b0d695705124ac0b5d0a63e70ad45ce/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:52:12.065998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907afbab-cd6a-4d4b-83d6-24fbcdc2eecf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:12.066634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907afbab-cd6a-4d4b-83d6-24fbcdc2eecf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:12.066634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907afbab-cd6a-4d4b-83d6-24fbcdc2eecf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:12.067917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 638/1274] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:52:12.074160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f779d-7bba-45c5-80c8-ecea76cd0b86 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:12.074917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f779d-7bba-45c5-80c8-ecea76cd0b86 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:12.075058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f779d-7bba-45c5-80c8-ecea76cd0b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:12.101039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-457f779d-7bba-45c5-80c8-ecea76cd0b86 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:52:12.102245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-457f779d-7bba-45c5-80c8-ecea76cd0b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:52:12.180960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 637/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:52:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:52:12.444560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907afbab-cd6a-4d4b-83d6-24fbcdc2eecf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:12.554881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907afbab-cd6a-4d4b-83d6-24fbcdc2eecf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:12.572384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:12.591586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:12.592303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:12.592587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:12.592840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:12.603839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 05 16:52:12.609086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:12.610346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:12.611616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 639/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Sep 5 16:52:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:12.616410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f779d-7bba-45c5-80c8-ecea76cd0b86 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:12.700069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:12.721972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f779d-7bba-45c5-80c8-ecea76cd0b86 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:12.756707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:12.757295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:12.757569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:12.757833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:12.758482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:12.758781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:12.760128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:12.768476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 638/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Sep 5 16:52:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 16:52:12.774858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:12.887241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af9006-8a51-43b3-ab0e-541a65926901 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:12.935394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:12.939882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:12.939882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:12.939882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:12.948123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:12.950245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:52:12.970547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:12.971737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:52:12.972741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:52:12.972741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:12.972741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:12.972741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 640/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:12.979145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:13.088758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efedfb-e04a-4084-9521-db3edb2519bd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:13.124758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:13.125358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:13.125630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:13.125899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:13.126524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:13.126827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:13.128191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:13.140528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 639/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:13.149017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:13.191628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:13.279247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047aacc2-273d-4c27-9efb-131c91fa53b1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:13.316587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:13.317217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:13.317494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:13.317767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:13.325590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:13.327252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 05 16:52:13.338627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:13.339736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:13.340897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 641/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:13.346034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:13.349993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:13.453436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed04e6-b836-4528-bcc5-0ce77918e6db None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:13.489322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:13.489925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:13.490199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:13.490468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:13.491191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:13.491495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:13.492954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:13.510163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 640/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Sep 5 16:52:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:13.519394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:13.634442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:13.644988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a5b55-6e37-4497-b834-2175dfb892b9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:13.680584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:13.681187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:13.681464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:13.681734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:13.753379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:13.754846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:13.852406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:13.865700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 642/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Sep 5 16:52:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 05 16:52:13.871921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:13.939664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:13.982165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82731d3c-c47b-47b1-aa0f-55b232c6dcd7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:14.026015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:14.026642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:14.026918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:14.027188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:14.039308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:14.040796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 05 16:52:14.056623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:14.060393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 05 16:52:14.061806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 05 16:52:14.061806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 05 16:52:14.061806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:14.061806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 641/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:14.063611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:14.178358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b5bba-4f39-448b-b41c-d9ffbaac952d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:14.216312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:14.216845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:14.217164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:14.217472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:14.218106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:14.218414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:14.219780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:14.231411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 643/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Sep 5 16:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:14.240848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:14.357153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6c2bd-e332-46ff-8987-20c77815fc38 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:14.371320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:14.394716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:14.395352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:14.395629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:14.395898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:14.402753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:14.404218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 05 16:52:14.415607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:14.416602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Sep 05 16:52:14.418007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 642/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Sep 5 16:52:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:14.423264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:14.477048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:14.529263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887c81d-9e21-42c4-aae2-29922069c26a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:14.566073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82f146-6100-4fcb-8c52-1c0ffff7b057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:14.566669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f146-6100-4fcb-8c52-1c0ffff7b057 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:14.566948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f146-6100-4fcb-8c52-1c0ffff7b057 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:14.567222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82f146-6100-4fcb-8c52-1c0ffff7b057 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:14.567934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82f146-6100-4fcb-8c52-1c0ffff7b057 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:14.568269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82f146-6100-4fcb-8c52-1c0ffff7b057 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:14.569717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82f146-6100-4fcb-8c52-1c0ffff7b057 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:14.582306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 644/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Sep 5 16:52:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:14.593256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:14.711168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18420c92-d87b-4e60-80da-39a386ab09be None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:14.746429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62970f22-ff3a-4809-b664-57c46a7760c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:14.746957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62970f22-ff3a-4809-b664-57c46a7760c0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:14.747235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62970f22-ff3a-4809-b664-57c46a7760c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:14.747503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62970f22-ff3a-4809-b664-57c46a7760c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:14.748223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62970f22-ff3a-4809-b664-57c46a7760c0 None admin] RBAC: Adding query filter params (role.id=d6ba822e8c4c4402961c6db999ff6fd1, scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1, user.id=2b0d695705124ac0b5d0a63e70ad45ce) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:14.748750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62970f22-ff3a-4809-b664-57c46a7760c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:14.750173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62970f22-ff3a-4809-b664-57c46a7760c0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:14.756482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 643/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Sep 5 16:52:14 2022] GET /identity/v3/role_assignments?user.id=2b0d695705124ac0b5d0a63e70ad45ce&scope.project.id=ba6bbf6ebf764fe08027e5e11c0e65c1&role.id=d6ba822e8c4c4402961c6db999ff6fd1 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:16.676067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d406f3-ad6b-4905-b9ac-4a29b089c8ea None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:16.676976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d406f3-ad6b-4905-b9ac-4a29b089c8ea None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:16.676976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d406f3-ad6b-4905-b9ac-4a29b089c8ea None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:16.679030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 645/1288] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:52:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:52:16.687688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47620-8f96-4874-85a4-d5bff80fc79e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:16.688082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47620-8f96-4874-85a4-d5bff80fc79e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:16.688259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d47620-8f96-4874-85a4-d5bff80fc79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:16.709492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-30d47620-8f96-4874-85a4-d5bff80fc79e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:52:16.710681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-30d47620-8f96-4874-85a4-d5bff80fc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:52:16.747184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:16.789818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 644/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:52:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:52:17.062645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d406f3-ad6b-4905-b9ac-4a29b089c8ea None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:17.171418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d406f3-ad6b-4905-b9ac-4a29b089c8ea None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:17.181939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:17.209364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:17.210002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:17.210280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:17.210547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:17.211107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:17.212505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:17.222518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:17.223605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:17.224648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 646/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:52:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:17.230265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47620-8f96-4874-85a4-d5bff80fc79e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:17.340623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d47620-8f96-4874-85a4-d5bff80fc79e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:17.378410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced03fb1-3d52-4877-ab4f-3510fa11c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:17.379027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced03fb1-3d52-4877-ab4f-3510fa11c551 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:17.379306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced03fb1-3d52-4877-ab4f-3510fa11c551 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:17.379576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced03fb1-3d52-4877-ab4f-3510fa11c551 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:17.380237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced03fb1-3d52-4877-ab4f-3510fa11c551 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:17.380542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced03fb1-3d52-4877-ab4f-3510fa11c551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:17.381910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced03fb1-3d52-4877-ab4f-3510fa11c551 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:17.386994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 645/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Sep 5 16:52:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:17.396008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:17.513041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81b709-43bf-42bc-a8fd-9811b63b8422 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:17.547398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ec507-c3e6-4971-83e0-80a34ae8aafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:17.547996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ec507-c3e6-4971-83e0-80a34ae8aafd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:17.548300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ec507-c3e6-4971-83e0-80a34ae8aafd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:17.548573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ec507-c3e6-4971-83e0-80a34ae8aafd None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:17.549224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ec507-c3e6-4971-83e0-80a34ae8aafd None admin] RBAC: Adding query filter params (type=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:17.549522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ec507-c3e6-4971-83e0-80a34ae8aafd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:17.550919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ec507-c3e6-4971-83e0-80a34ae8aafd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:17.555317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 647/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Sep 5 16:52:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:19.426625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bccf8-6dd4-44be-9c86-8ec5098e9b90 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:19.427472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bccf8-6dd4-44be-9c86-8ec5098e9b90 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:19.427472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527bccf8-6dd4-44be-9c86-8ec5098e9b90 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:19.429725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 646/1293] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:52:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:52:19.438355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7465b1-6c4c-4f23-9d77-7305f3074148 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:19.438794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7465b1-6c4c-4f23-9d77-7305f3074148 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:19.438897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7465b1-6c4c-4f23-9d77-7305f3074148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:19.467799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6b7465b1-6c4c-4f23-9d77-7305f3074148 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:52:19.469004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6b7465b1-6c4c-4f23-9d77-7305f3074148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:52:19.500181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:19.538668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 648/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:52:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:52:19.820208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bccf8-6dd4-44be-9c86-8ec5098e9b90 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:19.858395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:19.949976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527bccf8-6dd4-44be-9c86-8ec5098e9b90 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:20.004620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f087eb-913d-42b7-a240-5836e3988913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:20.005470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f087eb-913d-42b7-a240-5836e3988913 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:20.005956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f087eb-913d-42b7-a240-5836e3988913 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:20.006427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f087eb-913d-42b7-a240-5836e3988913 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:20.007888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f087eb-913d-42b7-a240-5836e3988913 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:20.009674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f087eb-913d-42b7-a240-5836e3988913 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:20.026792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 647/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Sep 5 16:52:19 2022] POST /identity/v3/services => generated 246 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:52:21.914275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8300-0729-453c-9a64-c61fe226d585 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:21.915180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8300-0729-453c-9a64-c61fe226d585 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:21.915180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d8300-0729-453c-9a64-c61fe226d585 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:21.917367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 649/1296] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:52:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:52:21.927016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4dcf8-7e81-48a9-9985-561c90d46018 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:21.927456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4dcf8-7e81-48a9-9985-561c90d46018 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:21.927717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4dcf8-7e81-48a9-9985-561c90d46018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:21.950269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ffd4dcf8-7e81-48a9-9985-561c90d46018 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:52:21.951618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ffd4dcf8-7e81-48a9-9985-561c90d46018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:52:22.031965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 648/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:52:21 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:52:22.298630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8300-0729-453c-9a64-c61fe226d585 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:22.420058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d8300-0729-453c-9a64-c61fe226d585 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:22.458053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:22.458705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:22.458984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:22.459251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:22.459829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:22.461241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:22.470970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:22.472163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:22.473024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 650/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:52:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:22.478262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4dcf8-7e81-48a9-9985-561c90d46018 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:22.497665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:22.594228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4dcf8-7e81-48a9-9985-561c90d46018 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:22.630066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:22.630662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:22.630935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:22.631205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:22.631824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:22.632127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:22.633512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:22.638694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 649/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Sep 5 16:52:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:22.645654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:22.683445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:22.764020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f218b46-dd24-46d0-adeb-9d52b39e7859 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:22.803066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b3469-81aa-4c3a-995d-f2deb0251c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:22.803659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b3469-81aa-4c3a-995d-f2deb0251c7d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:22.803936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b3469-81aa-4c3a-995d-f2deb0251c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:22.804211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b3469-81aa-4c3a-995d-f2deb0251c7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:22.804925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b3469-81aa-4c3a-995d-f2deb0251c7d None admin] RBAC: Adding query filter params (interface=public, service_id=2c2619434a3548a8b28dc70e64a10fa9, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:22.805232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b3469-81aa-4c3a-995d-f2deb0251c7d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:22.806607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b3469-81aa-4c3a-995d-f2deb0251c7d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:22.812004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 651/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Sep 5 16:52:22 2022] GET /identity/v3/endpoints?service_id=2c2619434a3548a8b28dc70e64a10fa9&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:22.817954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:22.902434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:22.924853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c15d7c-4008-47ae-ad1b-380687153d47 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:22.958823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91ff3d-f0f7-41f2-8c06-ddb3fa76b462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:22.959427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91ff3d-f0f7-41f2-8c06-ddb3fa76b462 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:22.959676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91ff3d-f0f7-41f2-8c06-ddb3fa76b462 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:22.959941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91ff3d-f0f7-41f2-8c06-ddb3fa76b462 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:22.960565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91ff3d-f0f7-41f2-8c06-ddb3fa76b462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:22.961917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91ff3d-f0f7-41f2-8c06-ddb3fa76b462 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:22.968815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 650/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Sep 5 16:52:22 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:52:24.839152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5c1d-b167-46ee-89fd-a1b2b7495174 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:24.840020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5c1d-b167-46ee-89fd-a1b2b7495174 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:24.840020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5c1d-b167-46ee-89fd-a1b2b7495174 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:24.842240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 652/1302] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:52:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:52:24.847803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277f51-c0b8-4636-98dd-c65e80d87d3c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:24.848226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277f51-c0b8-4636-98dd-c65e80d87d3c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:24.848400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277f51-c0b8-4636-98dd-c65e80d87d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:24.869495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-87277f51-c0b8-4636-98dd-c65e80d87d3c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:52:24.870695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-87277f51-c0b8-4636-98dd-c65e80d87d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:52:24.941829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 651/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:52:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 05 16:52:25.207028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a5c1d-b167-46ee-89fd-a1b2b7495174 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:25.326287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:25.335594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a5c1d-b167-46ee-89fd-a1b2b7495174 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:25.374199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:25.374840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:25.375113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:25.375378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:25.375955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:25.377382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 05 16:52:25.386797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 05 16:52:25.387868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Sep 05 16:52:25.388918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 653/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Sep 5 16:52:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 05 16:52:25.394392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87277f51-c0b8-4636-98dd-c65e80d87d3c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:25.473348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:25.501814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87277f51-c0b8-4636-98dd-c65e80d87d3c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:25.544349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ad0a4-3215-4789-b013-89b9ca711c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:25.544981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad0a4-3215-4789-b013-89b9ca711c2a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:25.545255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad0a4-3215-4789-b013-89b9ca711c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:25.545500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ad0a4-3215-4789-b013-89b9ca711c2a None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:25.546147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad0a4-3215-4789-b013-89b9ca711c2a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:52:25.546452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad0a4-3215-4789-b013-89b9ca711c2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:25.547818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ad0a4-3215-4789-b013-89b9ca711c2a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:25.553494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 652/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Sep 5 16:52:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:52:25.560273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:52:25.673399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3f44b-2533-47fa-9b18-e0d1e576213d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:52:25.710119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdb59e-e201-4114-8836-6f2eb3f3b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:52:25.710718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdb59e-e201-4114-8836-6f2eb3f3b938 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:52:25.710991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdb59e-e201-4114-8836-6f2eb3f3b938 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:52:25.711258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdb59e-e201-4114-8836-6f2eb3f3b938 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:52:25.711913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdb59e-e201-4114-8836-6f2eb3f3b938 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:52:25.713317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdb59e-e201-4114-8836-6f2eb3f3b938 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:52:25.737944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 654/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:52:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:52:35.550102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=30 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:52:35.733715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:30.161760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc269ca3-5554-462a-af85-51eb8b1b248d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:30.162342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc269ca3-5554-462a-af85-51eb8b1b248d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:30.162342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc269ca3-5554-462a-af85-51eb8b1b248d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:30.167440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 653/1307] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:30.168835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be10413-422a-4bb1-802b-6b3874254ff5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:30.169318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be10413-422a-4bb1-802b-6b3874254ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:30.169421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be10413-422a-4bb1-802b-6b3874254ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:30.311981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6be10413-422a-4bb1-802b-6b3874254ff5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:30.313278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6be10413-422a-4bb1-802b-6b3874254ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:30.534921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 655/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:53:30.849200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f99290-abc1-432f-999f-6c00c7189ad2 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:30.850190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f99290-abc1-432f-999f-6c00c7189ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:30.851135 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f99290-abc1-432f-999f-6c00c7189ad2 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:30.853572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 654/1309] 10.222.0.23 () {54 vars in 965 bytes} [Mon Sep 5 16:53:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:30.859684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991512aa-6a3d-4fe7-a426-b47a43df89ca None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:30.859952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991512aa-6a3d-4fe7-a426-b47a43df89ca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:30.860254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991512aa-6a3d-4fe7-a426-b47a43df89ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:31.020441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-991512aa-6a3d-4fe7-a426-b47a43df89ca None None] MFA Rules not processed for user `fcbcd7638cc84cdd87a1eb98f5926425`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:31.022057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-991512aa-6a3d-4fe7-a426-b47a43df89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:31.045216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:31.117695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 656/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Mon Sep 5 16:53:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:53:31.127573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f99290-abc1-432f-999f-6c00c7189ad2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:31.129359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a8f99290-abc1-432f-999f-6c00c7189ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:31.277352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f99290-abc1-432f-999f-6c00c7189ad2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:31.376745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb20759-fb30-4361-b6cc-3c43b5e1c148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:31.377390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb20759-fb30-4361-b6cc-3c43b5e1c148 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:31.377673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb20759-fb30-4361-b6cc-3c43b5e1c148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:31.377948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb20759-fb30-4361-b6cc-3c43b5e1c148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:31.379761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6fb20759-fb30-4361-b6cc-3c43b5e1c148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:31.453294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:31.515630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb20759-fb30-4361-b6cc-3c43b5e1c148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:31.517288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb20759-fb30-4361-b6cc-3c43b5e1c148 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:31.537760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 655/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:33.831248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9acd5-202b-4fae-8a85-76af20bc2477 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:33.831839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9acd5-202b-4fae-8a85-76af20bc2477 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:33.831839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9acd5-202b-4fae-8a85-76af20bc2477 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:33.833356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 657/1312] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:33.838642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db450-5312-480e-b17e-096199d5aeca None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:33.839089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db450-5312-480e-b17e-096199d5aeca None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:33.839184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db450-5312-480e-b17e-096199d5aeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:33.928411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ae0db450-5312-480e-b17e-096199d5aeca None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:33.929611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ae0db450-5312-480e-b17e-096199d5aeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:34.034751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 656/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:34.311775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9acd5-202b-4fae-8a85-76af20bc2477 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:34.313919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ccc9acd5-202b-4fae-8a85-76af20bc2477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:34.444556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9acd5-202b-4fae-8a85-76af20bc2477 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:34.532718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:34.535371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd54738-7dfe-4a89-b0f7-621a4cb846fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:34.536019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54738-7dfe-4a89-b0f7-621a4cb846fa service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:34.536326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54738-7dfe-4a89-b0f7-621a4cb846fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:34.536588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd54738-7dfe-4a89-b0f7-621a4cb846fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:34.539983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ddd54738-7dfe-4a89-b0f7-621a4cb846fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:34.656829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd54738-7dfe-4a89-b0f7-621a4cb846fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:34.658297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd54738-7dfe-4a89-b0f7-621a4cb846fa service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:34.676903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 658/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:36.893157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d9e7e-b218-48eb-b6bd-c6c9a36e401f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:36.894097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d9e7e-b218-48eb-b6bd-c6c9a36e401f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:36.894097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d9e7e-b218-48eb-b6bd-c6c9a36e401f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:36.896050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 657/1315] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:36.905194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed767bf1-6094-432e-a9dc-3d587c90f905 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:36.905921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed767bf1-6094-432e-a9dc-3d587c90f905 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:36.906069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed767bf1-6094-432e-a9dc-3d587c90f905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:36.981688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ed767bf1-6094-432e-a9dc-3d587c90f905 None None] MFA Rules not processed for user `e46173bc1d274512ae45ef02806797bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:36.982916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ed767bf1-6094-432e-a9dc-3d587c90f905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:37.040275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:37.089579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 659/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:37.398014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d9e7e-b218-48eb-b6bd-c6c9a36e401f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:37.400497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-140d9e7e-b218-48eb-b6bd-c6c9a36e401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:37.411635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:37.517873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d9e7e-b218-48eb-b6bd-c6c9a36e401f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:37.592573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaa9fe-cd19-4e34-973d-d758fd5ff553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:37.593187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaa9fe-cd19-4e34-973d-d758fd5ff553 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:37.593463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaa9fe-cd19-4e34-973d-d758fd5ff553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:37.593743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaa9fe-cd19-4e34-973d-d758fd5ff553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:37.595324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dbeaa9fe-cd19-4e34-973d-d758fd5ff553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:37.662746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:37.711055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeaa9fe-cd19-4e34-973d-d758fd5ff553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:37.712540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeaa9fe-cd19-4e34-973d-d758fd5ff553 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:37.732365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 658/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:40.169873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151769-e1ec-4d86-b6dc-c8913e7b5eea None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:40.170433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151769-e1ec-4d86-b6dc-c8913e7b5eea None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:40.170433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59151769-e1ec-4d86-b6dc-c8913e7b5eea None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:40.171784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 660/1318] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:53:40.178258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44810a-c6f3-46cf-a2f1-31ee13b20f75 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:40.178946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44810a-c6f3-46cf-a2f1-31ee13b20f75 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:40.179115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44810a-c6f3-46cf-a2f1-31ee13b20f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:40.248173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ea44810a-c6f3-46cf-a2f1-31ee13b20f75 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:40.249381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea44810a-c6f3-46cf-a2f1-31ee13b20f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:40.359661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 659/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:40.644457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151769-e1ec-4d86-b6dc-c8913e7b5eea None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:40.645997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-59151769-e1ec-4d86-b6dc-c8913e7b5eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:40.759016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59151769-e1ec-4d86-b6dc-c8913e7b5eea None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:40.841804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a95b61-c410-4f62-a293-eff939a95bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:40.842419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a95b61-c410-4f62-a293-eff939a95bac service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:40.842696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a95b61-c410-4f62-a293-eff939a95bac service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:40.842966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a95b61-c410-4f62-a293-eff939a95bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:40.844582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d8a95b61-c410-4f62-a293-eff939a95bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:40.852080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:40.961940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a95b61-c410-4f62-a293-eff939a95bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:40.963316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a95b61-c410-4f62-a293-eff939a95bac service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:40.982458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 661/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:53:43.539319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c3f3-b910-4a66-a463-1315a4cb8857 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:43.539940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c3f3-b910-4a66-a463-1315a4cb8857 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:43.539940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c3f3-b910-4a66-a463-1315a4cb8857 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:43.541256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 660/1321] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:43.546753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6edb-4c04-4aa6-ad30-1a519afe52e1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:43.547189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6edb-4c04-4aa6-ad30-1a519afe52e1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:43.547295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6edb-4c04-4aa6-ad30-1a519afe52e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:43.615566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a2dc6edb-4c04-4aa6-ad30-1a519afe52e1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:43.616808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a2dc6edb-4c04-4aa6-ad30-1a519afe52e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:43.676001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:43.720987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 662/1322] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:44.019317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63f74-7674-44c9-9d2b-02f084aeb506 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:44.019580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63f74-7674-44c9-9d2b-02f084aeb506 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:44.019858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63f74-7674-44c9-9d2b-02f084aeb506 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:44.021286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 661/1323] 10.222.0.23 () {54 vars in 965 bytes} [Mon Sep 5 16:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:44.027521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f05d9-dcba-40d6-955b-f61719c9c7b0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:44.027839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f05d9-dcba-40d6-955b-f61719c9c7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:44.027936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f05d9-dcba-40d6-955b-f61719c9c7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:44.105905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-254f05d9-dcba-40d6-955b-f61719c9c7b0 None None] MFA Rules not processed for user `fcbcd7638cc84cdd87a1eb98f5926425`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:44.107136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-254f05d9-dcba-40d6-955b-f61719c9c7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:44.189954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 663/1324] 10.222.0.23 () {58 vars in 1062 bytes} [Mon Sep 5 16:53:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:44.199664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d63f74-7674-44c9-9d2b-02f084aeb506 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:44.201343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e9d63f74-7674-44c9-9d2b-02f084aeb506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:44.222228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:44.313131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d63f74-7674-44c9-9d2b-02f084aeb506 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:44.396552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770de107-a4da-4d43-b970-ac0abd16df55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:44.397169 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770de107-a4da-4d43-b970-ac0abd16df55 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:44.397447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770de107-a4da-4d43-b970-ac0abd16df55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:44.397718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770de107-a4da-4d43-b970-ac0abd16df55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:44.399311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-770de107-a4da-4d43-b970-ac0abd16df55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:44.478151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:44.518979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770de107-a4da-4d43-b970-ac0abd16df55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:44.520461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770de107-a4da-4d43-b970-ac0abd16df55 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:44.539997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 662/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:46.700943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52faba0-52c6-4f7c-bf6f-1bc543c3b8a2 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:46.701475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52faba0-52c6-4f7c-bf6f-1bc543c3b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:46.701475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52faba0-52c6-4f7c-bf6f-1bc543c3b8a2 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:46.703082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 664/1326] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:46.708544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34455b-6092-4cf4-960f-aaf514e39f03 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:46.708974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34455b-6092-4cf4-960f-aaf514e39f03 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:46.709077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34455b-6092-4cf4-960f-aaf514e39f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:46.779015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7c34455b-6092-4cf4-960f-aaf514e39f03 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:46.780235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c34455b-6092-4cf4-960f-aaf514e39f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:46.890693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 663/1327] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:47.179825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52faba0-52c6-4f7c-bf6f-1bc543c3b8a2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:47.181378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d52faba0-52c6-4f7c-bf6f-1bc543c3b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:47.247111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:47.293938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52faba0-52c6-4f7c-bf6f-1bc543c3b8a2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:47.373428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e8fba-96ca-4e6a-ad37-5abb0ad026c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:47.374052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8fba-96ca-4e6a-ad37-5abb0ad026c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:47.374439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8fba-96ca-4e6a-ad37-5abb0ad026c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:47.374724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e8fba-96ca-4e6a-ad37-5abb0ad026c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:47.376421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1d8e8fba-96ca-4e6a-ad37-5abb0ad026c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:47.492012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:47.494261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e8fba-96ca-4e6a-ad37-5abb0ad026c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:47.495833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e8fba-96ca-4e6a-ad37-5abb0ad026c4 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:47.514910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 665/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:49.484672 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ffe2-9eb1-4240-a01b-c23a78bbd7bb None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:49.485186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ffe2-9eb1-4240-a01b-c23a78bbd7bb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:49.485186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4ffe2-9eb1-4240-a01b-c23a78bbd7bb None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:49.486569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 664/1329] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:49.492278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad374219-a1db-4a26-8835-d3058d19b71e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:49.492780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad374219-a1db-4a26-8835-d3058d19b71e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:49.492876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad374219-a1db-4a26-8835-d3058d19b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:49.567605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ad374219-a1db-4a26-8835-d3058d19b71e None None] MFA Rules not processed for user `e46173bc1d274512ae45ef02806797bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:49.568864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ad374219-a1db-4a26-8835-d3058d19b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:49.667162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 666/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:49.962569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4ffe2-9eb1-4240-a01b-c23a78bbd7bb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:49.964081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c5c4ffe2-9eb1-4240-a01b-c23a78bbd7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:50.000048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:50.087232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4ffe2-9eb1-4240-a01b-c23a78bbd7bb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:50.173070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7eeef-3c46-4981-abe9-1432bcd567da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:50.173804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7eeef-3c46-4981-abe9-1432bcd567da service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:50.174140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7eeef-3c46-4981-abe9-1432bcd567da service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:50.174477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7eeef-3c46-4981-abe9-1432bcd567da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:50.176284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4ad7eeef-3c46-4981-abe9-1432bcd567da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:50.267468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:50.302441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7eeef-3c46-4981-abe9-1432bcd567da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:50.304321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7eeef-3c46-4981-abe9-1432bcd567da service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:50.323093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 665/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:52.617764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90d8c3-e465-43cc-9a37-d918bfc59fb8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:52.618324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90d8c3-e465-43cc-9a37-d918bfc59fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:52.618324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90d8c3-e465-43cc-9a37-d918bfc59fb8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:52.619686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 667/1332] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:52.626360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6d32-182b-44a2-9d06-0e4a05078fd9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:52.627069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6d32-182b-44a2-9d06-0e4a05078fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:52.627241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d6d32-182b-44a2-9d06-0e4a05078fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:52.697959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c65d6d32-182b-44a2-9d06-0e4a05078fd9 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:52.699325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c65d6d32-182b-44a2-9d06-0e4a05078fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:52.801172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 666/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:53.082161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90d8c3-e465-43cc-9a37-d918bfc59fb8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:53.084446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0b90d8c3-e465-43cc-9a37-d918bfc59fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:53.174912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:53.211813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90d8c3-e465-43cc-9a37-d918bfc59fb8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:53.294876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5ac7-06f4-4712-9ceb-03bebb56dca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:53.295487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5ac7-06f4-4712-9ceb-03bebb56dca2 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:53.295766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5ac7-06f4-4712-9ceb-03bebb56dca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:53.296049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5ac7-06f4-4712-9ceb-03bebb56dca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:53.297689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-66eb5ac7-06f4-4712-9ceb-03bebb56dca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:53.413269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb5ac7-06f4-4712-9ceb-03bebb56dca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:53.414714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb5ac7-06f4-4712-9ceb-03bebb56dca2 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:53.425638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:53.436128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 668/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:55.382399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8a7b-570f-4a2b-a8c2-55f01339eee7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:55.382885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8a7b-570f-4a2b-a8c2-55f01339eee7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:55.382885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea8a7b-570f-4a2b-a8c2-55f01339eee7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:55.384320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 667/1335] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:55.389564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f318-0508-412f-8dfb-d1e9df4ac036 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:55.390012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f318-0508-412f-8dfb-d1e9df4ac036 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:55.390115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f318-0508-412f-8dfb-d1e9df4ac036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:55.466347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5dc7f318-0508-412f-8dfb-d1e9df4ac036 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:55.467560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5dc7f318-0508-412f-8dfb-d1e9df4ac036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:55.566384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 669/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:55.855954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8a7b-570f-4a2b-a8c2-55f01339eee7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:55.859255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3ea8a7b-570f-4a2b-a8c2-55f01339eee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:55.904769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:55.987110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea8a7b-570f-4a2b-a8c2-55f01339eee7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:56.083637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f9ced-7289-46b8-88dc-726e40c1e5d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:56.084283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f9ced-7289-46b8-88dc-726e40c1e5d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:56.084558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f9ced-7289-46b8-88dc-726e40c1e5d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:56.084828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f9ced-7289-46b8-88dc-726e40c1e5d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:56.086422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a82f9ced-7289-46b8-88dc-726e40c1e5d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:56.194226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:56.221982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f9ced-7289-46b8-88dc-726e40c1e5d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:56.223427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f9ced-7289-46b8-88dc-726e40c1e5d6 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:56.242334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 668/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:53:58.599216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301a0ed-b40c-489d-b7a2-990cd93361cc None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:58.599816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301a0ed-b40c-489d-b7a2-990cd93361cc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:58.599816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301a0ed-b40c-489d-b7a2-990cd93361cc None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:58.601160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 670/1338] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:53:58.607969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc022da-4f30-4942-9ced-e5b0bc87148e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:58.608750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc022da-4f30-4942-9ced-e5b0bc87148e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:58.608943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc022da-4f30-4942-9ced-e5b0bc87148e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:58.697659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-abc022da-4f30-4942-9ced-e5b0bc87148e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:53:58.699198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-abc022da-4f30-4942-9ced-e5b0bc87148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:58.799124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 669/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:53:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:53:59.099246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301a0ed-b40c-489d-b7a2-990cd93361cc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:53:59.101771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1301a0ed-b40c-489d-b7a2-990cd93361cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:59.188178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:53:59.217362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301a0ed-b40c-489d-b7a2-990cd93361cc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:53:59.296005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11eed8-aad7-4240-ad99-4231e0668800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:53:59.296911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11eed8-aad7-4240-ad99-4231e0668800 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:53:59.297196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11eed8-aad7-4240-ad99-4231e0668800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:53:59.297463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11eed8-aad7-4240-ad99-4231e0668800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:53:59.299089 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fc11eed8-aad7-4240-ad99-4231e0668800 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:53:59.418204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11eed8-aad7-4240-ad99-4231e0668800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:53:59.419666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11eed8-aad7-4240-ad99-4231e0668800 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:53:59.438189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 671/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:53:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:03.385018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f723e-57e3-4016-8754-f2e20c49b31a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:03.385509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f723e-57e3-4016-8754-f2e20c49b31a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:03.385597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f723e-57e3-4016-8754-f2e20c49b31a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:03.387284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 670/1341] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:03.393359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870a6b4-2dcc-4ff1-b678-c814d56b5ae0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:03.393809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870a6b4-2dcc-4ff1-b678-c814d56b5ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:03.393902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870a6b4-2dcc-4ff1-b678-c814d56b5ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:03.402109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:03.472901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d870a6b4-2dcc-4ff1-b678-c814d56b5ae0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:03.474680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d870a6b4-2dcc-4ff1-b678-c814d56b5ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:03.572795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 672/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:03.852612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f723e-57e3-4016-8754-f2e20c49b31a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:03.854255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-063f723e-57e3-4016-8754-f2e20c49b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:03.904554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:03.962476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f723e-57e3-4016-8754-f2e20c49b31a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:04.042176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e78d5-9310-4e64-8de0-acbadbb9701a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:04.042909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e78d5-9310-4e64-8de0-acbadbb9701a service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:04.043184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e78d5-9310-4e64-8de0-acbadbb9701a service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:04.043428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e78d5-9310-4e64-8de0-acbadbb9701a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:04.045030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-388e78d5-9310-4e64-8de0-acbadbb9701a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:04.140983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:04.158076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e78d5-9310-4e64-8de0-acbadbb9701a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:04.159515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e78d5-9310-4e64-8de0-acbadbb9701a service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:04.179509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 671/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:06.740637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da6e69-bf45-4d55-a9a1-3774ebe1a2d4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:06.741182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da6e69-bf45-4d55-a9a1-3774ebe1a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:06.741182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da6e69-bf45-4d55-a9a1-3774ebe1a2d4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:06.742474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 673/1344] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:06.749241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bf85f-a41b-4c05-8d59-8c85809f2943 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:06.749815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bf85f-a41b-4c05-8d59-8c85809f2943 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:06.749815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bf85f-a41b-4c05-8d59-8c85809f2943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:06.829963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3a3bf85f-a41b-4c05-8d59-8c85809f2943 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:06.831250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3a3bf85f-a41b-4c05-8d59-8c85809f2943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:06.936249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 672/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:07.225252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da6e69-bf45-4d55-a9a1-3774ebe1a2d4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:07.226829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-04da6e69-bf45-4d55-a9a1-3774ebe1a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:07.318612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:07.336501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da6e69-bf45-4d55-a9a1-3774ebe1a2d4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:07.413978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800b496-10ce-4714-9321-9f266af7943a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:07.414594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800b496-10ce-4714-9321-9f266af7943a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:07.414868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800b496-10ce-4714-9321-9f266af7943a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:07.415138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800b496-10ce-4714-9321-9f266af7943a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:07.416835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c800b496-10ce-4714-9321-9f266af7943a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:07.545634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800b496-10ce-4714-9321-9f266af7943a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:07.547006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800b496-10ce-4714-9321-9f266af7943a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:07.565771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 674/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:11.462663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5102647-6cb9-43eb-a4a4-229b84f785e1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:11.463180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5102647-6cb9-43eb-a4a4-229b84f785e1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:11.463304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5102647-6cb9-43eb-a4a4-229b84f785e1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:11.464667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 673/1347] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:11.470184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c125e-229a-4340-8e33-22edc3ee9af4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:11.470604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c125e-229a-4340-8e33-22edc3ee9af4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:11.470943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c125e-229a-4340-8e33-22edc3ee9af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:11.488105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:11.555704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c74c125e-229a-4340-8e33-22edc3ee9af4 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:11.556974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c74c125e-229a-4340-8e33-22edc3ee9af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:11.667980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 675/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:11.947225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5102647-6cb9-43eb-a4a4-229b84f785e1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:11.949224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a5102647-6cb9-43eb-a4a4-229b84f785e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:12.010678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:12.067065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5102647-6cb9-43eb-a4a4-229b84f785e1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:12.156301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af68ab4-2690-497b-a442-8f5692875804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:12.156925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68ab4-2690-497b-a442-8f5692875804 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:12.157200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68ab4-2690-497b-a442-8f5692875804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:12.157466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68ab4-2690-497b-a442-8f5692875804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:12.159058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5af68ab4-2690-497b-a442-8f5692875804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:12.269477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:12.273399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af68ab4-2690-497b-a442-8f5692875804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:12.274846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af68ab4-2690-497b-a442-8f5692875804 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:12.294438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 674/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:17.258237 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0dfed-3da7-40d3-af8e-11ef19f6bb3b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:17.258733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0dfed-3da7-40d3-af8e-11ef19f6bb3b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:17.258831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0dfed-3da7-40d3-af8e-11ef19f6bb3b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:17.260129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 676/1350] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 16:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:17.265430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1221a4-58f8-4fb7-a80a-502e2b67d3ac None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:17.265910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1221a4-58f8-4fb7-a80a-502e2b67d3ac None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:17.266014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1221a4-58f8-4fb7-a80a-502e2b67d3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:17.401661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8b1221a4-58f8-4fb7-a80a-502e2b67d3ac None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:17.403420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8b1221a4-58f8-4fb7-a80a-502e2b67d3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:17.544962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 675/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 16:54:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:21.659028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42528b-aa61-4e4b-8eea-34a4fbd871f4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:21.659964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42528b-aa61-4e4b-8eea-34a4fbd871f4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:21.660520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42528b-aa61-4e4b-8eea-34a4fbd871f4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:21.661979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 677/1352] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 16:54:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:21.665839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=59 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:21.667317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105d5de-7ab9-4328-8a5f-5324e2ff3dc4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:21.667742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105d5de-7ab9-4328-8a5f-5324e2ff3dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:21.668065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105d5de-7ab9-4328-8a5f-5324e2ff3dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:21.725038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:21.751607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4105d5de-7ab9-4328-8a5f-5324e2ff3dc4 None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:21.752861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4105d5de-7ab9-4328-8a5f-5324e2ff3dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:21.864581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 676/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 16:54:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:23.226689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26c69a-7f68-431b-a1d2-d72330a3f47c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:23.227266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26c69a-7f68-431b-a1d2-d72330a3f47c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:23.227266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26c69a-7f68-431b-a1d2-d72330a3f47c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:23.228825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 678/1354] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 16:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:23.234830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2285c-b5e9-4228-a565-1a0540bc0d17 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:23.235294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2285c-b5e9-4228-a565-1a0540bc0d17 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:23.235409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2285c-b5e9-4228-a565-1a0540bc0d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:23.317517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-abd2285c-b5e9-4228-a565-1a0540bc0d17 None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:23.318748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-abd2285c-b5e9-4228-a565-1a0540bc0d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:23.363597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:23.435919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 677/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 16:54:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:26.675955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c2b0-877b-437b-8353-db304a5a5039 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:26.676522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c2b0-877b-437b-8353-db304a5a5039 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:26.676522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c2b0-877b-437b-8353-db304a5a5039 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:26.677843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 679/1356] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:26.683162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34633895-a48a-4e33-a658-d6b7a9b77c49 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:26.683634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34633895-a48a-4e33-a658-d6b7a9b77c49 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:26.683730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34633895-a48a-4e33-a658-d6b7a9b77c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:26.765150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-34633895-a48a-4e33-a658-d6b7a9b77c49 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:26.766366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-34633895-a48a-4e33-a658-d6b7a9b77c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:26.898212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 678/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:27.197869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f48c7-d9be-49d6-be2b-118e9140092c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:27.198753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f48c7-d9be-49d6-be2b-118e9140092c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:27.199443 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f48c7-d9be-49d6-be2b-118e9140092c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:27.201768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 680/1358] 10.222.0.23 () {54 vars in 962 bytes} [Mon Sep 5 16:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:27.218124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bf24-0ad3-4452-9536-a8a2000434d2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:27.218843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bf24-0ad3-4452-9536-a8a2000434d2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:27.220031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bf24-0ad3-4452-9536-a8a2000434d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:27.237159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:27.302179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e0f9bf24-0ad3-4452-9536-a8a2000434d2 None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:27.303713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e0f9bf24-0ad3-4452-9536-a8a2000434d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:27.455117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 679/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Mon Sep 5 16:54:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:27.463431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f48c7-d9be-49d6-be2b-118e9140092c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:27.465313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-815f48c7-d9be-49d6-be2b-118e9140092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:27.630272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f48c7-d9be-49d6-be2b-118e9140092c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:27.709142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:27.750349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154a7a2-bbae-423d-976c-a9edd3f949c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:27.750968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154a7a2-bbae-423d-976c-a9edd3f949c4 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:27.751248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154a7a2-bbae-423d-976c-a9edd3f949c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:27.751521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154a7a2-bbae-423d-976c-a9edd3f949c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:27.753161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f154a7a2-bbae-423d-976c-a9edd3f949c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:27.850087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154a7a2-bbae-423d-976c-a9edd3f949c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:27.851725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154a7a2-bbae-423d-976c-a9edd3f949c4 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:27.870953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 681/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:54:29.973617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba590981-e8c9-4f85-8e77-645baabacf91 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:29.974142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba590981-e8c9-4f85-8e77-645baabacf91 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:29.974142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba590981-e8c9-4f85-8e77-645baabacf91 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:29.975510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 680/1361] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:29.981318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4fd0a-b938-4408-b7c2-e9155af3ec10 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:29.981784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4fd0a-b938-4408-b7c2-e9155af3ec10 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:29.981883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4fd0a-b938-4408-b7c2-e9155af3ec10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:30.054149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e5b4fd0a-b938-4408-b7c2-e9155af3ec10 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:30.055357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e5b4fd0a-b938-4408-b7c2-e9155af3ec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:30.096076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:30.168328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 682/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:30.454185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba590981-e8c9-4f85-8e77-645baabacf91 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:30.456442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ba590981-e8c9-4f85-8e77-645baabacf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:30.526649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:30.610972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba590981-e8c9-4f85-8e77-645baabacf91 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:30.730927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c861cd3b-7925-462a-8fdb-b9b24819103e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:30.731537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861cd3b-7925-462a-8fdb-b9b24819103e service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:30.731808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861cd3b-7925-462a-8fdb-b9b24819103e service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:30.732073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c861cd3b-7925-462a-8fdb-b9b24819103e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:30.733761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c861cd3b-7925-462a-8fdb-b9b24819103e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:30.758772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:30.836637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861cd3b-7925-462a-8fdb-b9b24819103e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:30.838482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c861cd3b-7925-462a-8fdb-b9b24819103e service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:30.861143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 681/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:33.078174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7addb78-6adb-4351-8872-24aae965dd37 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:33.079207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7addb78-6adb-4351-8872-24aae965dd37 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:33.079565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7addb78-6adb-4351-8872-24aae965dd37 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:33.081336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 683/1364] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:33.086765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389e7c-8f19-4e94-80f6-e3a9a5fde25c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:33.087295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389e7c-8f19-4e94-80f6-e3a9a5fde25c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:33.087295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389e7c-8f19-4e94-80f6-e3a9a5fde25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:33.166231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-43389e7c-8f19-4e94-80f6-e3a9a5fde25c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:33.167638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-43389e7c-8f19-4e94-80f6-e3a9a5fde25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:33.276486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:33.286301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 682/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:33.576400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59892a00-e52f-4ba2-b146-871635c179de None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:33.576954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59892a00-e52f-4ba2-b146-871635c179de None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:33.577434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59892a00-e52f-4ba2-b146-871635c179de None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:33.578872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 684/1366] 10.222.0.23 () {54 vars in 962 bytes} [Mon Sep 5 16:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:33.584814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b489bc-d531-4160-bb72-80565e818b46 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:33.585063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b489bc-d531-4160-bb72-80565e818b46 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:33.585369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b489bc-d531-4160-bb72-80565e818b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:33.673620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a1b489bc-d531-4160-bb72-80565e818b46 None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:33.674886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a1b489bc-d531-4160-bb72-80565e818b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:33.861841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 683/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Mon Sep 5 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:33.871084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59892a00-e52f-4ba2-b146-871635c179de None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:33.872971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-59892a00-e52f-4ba2-b146-871635c179de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:34.043528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59892a00-e52f-4ba2-b146-871635c179de None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:34.072422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:34.179682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34a858-eb70-421a-a52a-21e60a24bb67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:34.180412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34a858-eb70-421a-a52a-21e60a24bb67 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:34.180651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34a858-eb70-421a-a52a-21e60a24bb67 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:34.180865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34a858-eb70-421a-a52a-21e60a24bb67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:34.182563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af34a858-eb70-421a-a52a-21e60a24bb67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:34.304787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34a858-eb70-421a-a52a-21e60a24bb67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:34.306574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34a858-eb70-421a-a52a-21e60a24bb67 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:34.328706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 685/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:34.861420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173268-5094-4f5d-8672-5bf2d47bb412 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:34.862002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173268-5094-4f5d-8672-5bf2d47bb412 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:34.862002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173268-5094-4f5d-8672-5bf2d47bb412 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:34.863360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 684/1369] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:34.868983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fe32d-f699-48d7-83e8-3004004b0113 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:34.869269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fe32d-f699-48d7-83e8-3004004b0113 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:34.869580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fe32d-f699-48d7-83e8-3004004b0113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:34.903232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:34.944726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-de2fe32d-f699-48d7-83e8-3004004b0113 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:34.945945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-de2fe32d-f699-48d7-83e8-3004004b0113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:35.049645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 686/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:35.627678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732684d-3f58-4ab7-b16d-eff97d84694c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:35.627921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732684d-3f58-4ab7-b16d-eff97d84694c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:35.628232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732684d-3f58-4ab7-b16d-eff97d84694c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:35.631416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 685/1371] 10.222.0.23 () {54 vars in 964 bytes} [Mon Sep 5 16:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:35.635433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd3288-d6c2-4e35-80a1-8d977f8ddbdf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:35.635894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd3288-d6c2-4e35-80a1-8d977f8ddbdf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:35.635987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd3288-d6c2-4e35-80a1-8d977f8ddbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:35.715375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-80fd3288-d6c2-4e35-80a1-8d977f8ddbdf None None] MFA Rules not processed for user `36c795d7c6bc4c519c15f32489c44583`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:35.717295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-80fd3288-d6c2-4e35-80a1-8d977f8ddbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:35.766369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:35.798356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 687/1372] 10.222.0.23 () {58 vars in 1061 bytes} [Mon Sep 5 16:54:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:35.806019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732684d-3f58-4ab7-b16d-eff97d84694c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:35.807587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a732684d-3f58-4ab7-b16d-eff97d84694c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:35.861605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:35.920522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732684d-3f58-4ab7-b16d-eff97d84694c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:36.004469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f13ee-271a-4001-9c61-c7364c3b672f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:36.005144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f13ee-271a-4001-9c61-c7364c3b672f service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:36.005366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f13ee-271a-4001-9c61-c7364c3b672f service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:36.005604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f13ee-271a-4001-9c61-c7364c3b672f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:36.007454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a34f13ee-271a-4001-9c61-c7364c3b672f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:36.120397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:36.136589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f13ee-271a-4001-9c61-c7364c3b672f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:36.138023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f13ee-271a-4001-9c61-c7364c3b672f service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:36.160475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 686/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:36.316415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d1ab6-1175-418e-a894-104569179c9a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:36.316653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d1ab6-1175-418e-a894-104569179c9a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:36.317223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d1ab6-1175-418e-a894-104569179c9a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:36.318505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 688/1374] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:36.323982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf44ae-b3b4-4774-ac97-e1df1fb19466 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:36.324268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf44ae-b3b4-4774-ac97-e1df1fb19466 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:36.324565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf44ae-b3b4-4774-ac97-e1df1fb19466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:36.391688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-12cf44ae-b3b4-4774-ac97-e1df1fb19466 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:36.392980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-12cf44ae-b3b4-4774-ac97-e1df1fb19466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:36.492655 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 687/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:36.780182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d1ab6-1175-418e-a894-104569179c9a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:36.781847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d52d1ab6-1175-418e-a894-104569179c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:36.943177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d1ab6-1175-418e-a894-104569179c9a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:36.998798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:37.068966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b3d02-14e6-4408-9e52-7c1c6f96eed2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:37.069488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b3d02-14e6-4408-9e52-7c1c6f96eed2 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:37.069774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b3d02-14e6-4408-9e52-7c1c6f96eed2 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:37.070045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b3d02-14e6-4408-9e52-7c1c6f96eed2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:37.071636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2a8b3d02-14e6-4408-9e52-7c1c6f96eed2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:37.164842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b3d02-14e6-4408-9e52-7c1c6f96eed2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:37.166494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b3d02-14e6-4408-9e52-7c1c6f96eed2 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:37.185376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 689/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:39.108116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825a05d-1192-43d4-a5ca-b3a7a937b3e5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:39.108592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825a05d-1192-43d4-a5ca-b3a7a937b3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:39.108592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825a05d-1192-43d4-a5ca-b3a7a937b3e5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:39.109826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 688/1377] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:39.115412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc06838-5489-40ec-af72-492d728eab58 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:39.115806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc06838-5489-40ec-af72-492d728eab58 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:39.115985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc06838-5489-40ec-af72-492d728eab58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:39.193441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cdc06838-5489-40ec-af72-492d728eab58 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:39.194815 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cdc06838-5489-40ec-af72-492d728eab58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:39.204496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:39.314092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 690/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:54:39.616776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825a05d-1192-43d4-a5ca-b3a7a937b3e5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:39.618452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1825a05d-1192-43d4-a5ca-b3a7a937b3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:39.716473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:39.829813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825a05d-1192-43d4-a5ca-b3a7a937b3e5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:39.957046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f44e7-826f-493f-adfe-586e42c916ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:39.957571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f44e7-826f-493f-adfe-586e42c916ef service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:39.957850 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f44e7-826f-493f-adfe-586e42c916ef service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:39.958118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f44e7-826f-493f-adfe-586e42c916ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:39.959684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8d7f44e7-826f-493f-adfe-586e42c916ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:39.982572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:40.067768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f44e7-826f-493f-adfe-586e42c916ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:40.069596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7f44e7-826f-493f-adfe-586e42c916ef service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:40.092795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 689/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:42.120910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f62c-13f6-4a00-9f78-541721f1a20b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:42.121494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f62c-13f6-4a00-9f78-541721f1a20b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:42.121494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f62c-13f6-4a00-9f78-541721f1a20b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:42.125323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 691/1380] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:42.131369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307aeba-0801-40ef-9209-6a5117ea30fd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:42.131899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307aeba-0801-40ef-9209-6a5117ea30fd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:42.132127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307aeba-0801-40ef-9209-6a5117ea30fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:42.203291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f307aeba-0801-40ef-9209-6a5117ea30fd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:42.204583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f307aeba-0801-40ef-9209-6a5117ea30fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:42.303047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:42.310765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 690/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:42.598734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f62c-13f6-4a00-9f78-541721f1a20b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:42.600396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a05f62c-13f6-4a00-9f78-541721f1a20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:42.753168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:42.767066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f62c-13f6-4a00-9f78-541721f1a20b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:42.896736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65abb44a-12ff-4870-8b8e-1aea80191c72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:42.897263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abb44a-12ff-4870-8b8e-1aea80191c72 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:42.897661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abb44a-12ff-4870-8b8e-1aea80191c72 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:42.897945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65abb44a-12ff-4870-8b8e-1aea80191c72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:42.899807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65abb44a-12ff-4870-8b8e-1aea80191c72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:43.005953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abb44a-12ff-4870-8b8e-1aea80191c72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:43.007820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65abb44a-12ff-4870-8b8e-1aea80191c72 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:43.020061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:43.030297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 692/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:45.035517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9933c1-8f81-4eb7-bf84-d3a7ad13bff6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:45.036243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9933c1-8f81-4eb7-bf84-d3a7ad13bff6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:45.036243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9933c1-8f81-4eb7-bf84-d3a7ad13bff6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:45.037452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 691/1383] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:45.043230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfdd8-f05a-4450-add4-131c664bed1a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:45.043760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfdd8-f05a-4450-add4-131c664bed1a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:45.043760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dfdd8-f05a-4450-add4-131c664bed1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:45.126356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-977dfdd8-f05a-4450-add4-131c664bed1a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:45.128320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-977dfdd8-f05a-4450-add4-131c664bed1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:45.253688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 693/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:45.544071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9933c1-8f81-4eb7-bf84-d3a7ad13bff6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:45.545611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6b9933c1-8f81-4eb7-bf84-d3a7ad13bff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:45.698499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9933c1-8f81-4eb7-bf84-d3a7ad13bff6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:45.778420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:45.830804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f7e8e-5fdd-4eaa-ae47-9e132d52f080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:45.831442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7e8e-5fdd-4eaa-ae47-9e132d52f080 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:45.831740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7e8e-5fdd-4eaa-ae47-9e132d52f080 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:45.832021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7e8e-5fdd-4eaa-ae47-9e132d52f080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:45.833726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-300f7e8e-5fdd-4eaa-ae47-9e132d52f080 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:45.929932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f7e8e-5fdd-4eaa-ae47-9e132d52f080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:45.932060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f7e8e-5fdd-4eaa-ae47-9e132d52f080 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:45.950926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 692/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:48.091063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd3c21-c70a-4485-a0bd-884309b0f888 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:48.091970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd3c21-c70a-4485-a0bd-884309b0f888 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:48.091970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd3c21-c70a-4485-a0bd-884309b0f888 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:48.093162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 694/1386] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:48.099208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbe970-0326-4358-91b4-aff46c31bbfe None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:48.100116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbe970-0326-4358-91b4-aff46c31bbfe None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:48.100466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbe970-0326-4358-91b4-aff46c31bbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:48.191958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ffcbe970-0326-4358-91b4-aff46c31bbfe None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:48.194566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ffcbe970-0326-4358-91b4-aff46c31bbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:48.367925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:48.454208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 693/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:54:48.742057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd3c21-c70a-4485-a0bd-884309b0f888 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:48.745093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5fcd3c21-c70a-4485-a0bd-884309b0f888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:48.881039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:49.002328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd3c21-c70a-4485-a0bd-884309b0f888 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:49.132038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efc2e9-b842-4c41-9ea3-834e6a4219ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:49.132786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efc2e9-b842-4c41-9ea3-834e6a4219ff service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:49.132932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efc2e9-b842-4c41-9ea3-834e6a4219ff service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:49.133231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efc2e9-b842-4c41-9ea3-834e6a4219ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:49.134843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e9efc2e9-b842-4c41-9ea3-834e6a4219ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:49.191004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:49.271085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efc2e9-b842-4c41-9ea3-834e6a4219ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:49.272673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efc2e9-b842-4c41-9ea3-834e6a4219ff service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:49.292321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 695/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:51.234903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac415b-84d4-4887-8584-19aba10f9816 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:51.235508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac415b-84d4-4887-8584-19aba10f9816 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:51.235508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac415b-84d4-4887-8584-19aba10f9816 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:51.236831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 694/1389] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:51.242288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f062cd3-7936-40a0-84b2-9a18a408dfc5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:51.242762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f062cd3-7936-40a0-84b2-9a18a408dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:51.242870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f062cd3-7936-40a0-84b2-9a18a408dfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:51.313212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8f062cd3-7936-40a0-84b2-9a18a408dfc5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:51.314415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8f062cd3-7936-40a0-84b2-9a18a408dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:51.419064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 696/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:51.701378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac415b-84d4-4887-8584-19aba10f9816 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:51.702867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-48ac415b-84d4-4887-8584-19aba10f9816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:51.855229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac415b-84d4-4887-8584-19aba10f9816 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:51.873016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:51.978047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982294c8-de8b-49cc-85ff-afedb64fc51b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:51.978670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982294c8-de8b-49cc-85ff-afedb64fc51b service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:51.979038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982294c8-de8b-49cc-85ff-afedb64fc51b service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:51.979608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982294c8-de8b-49cc-85ff-afedb64fc51b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:51.981208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-982294c8-de8b-49cc-85ff-afedb64fc51b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:52.073610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982294c8-de8b-49cc-85ff-afedb64fc51b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:52.074684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982294c8-de8b-49cc-85ff-afedb64fc51b service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:52.093599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 695/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:52.268376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1343a-c1e1-497a-9809-d3e001c4da83 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:52.268928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1343a-c1e1-497a-9809-d3e001c4da83 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:52.268928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1343a-c1e1-497a-9809-d3e001c4da83 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:52.270260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 697/1392] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:52.276612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1245dd1-06a6-4031-8866-f6230826d291 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:52.277206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1245dd1-06a6-4031-8866-f6230826d291 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:52.277206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1245dd1-06a6-4031-8866-f6230826d291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:52.305425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:52.349024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b1245dd1-06a6-4031-8866-f6230826d291 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:52.350224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b1245dd1-06a6-4031-8866-f6230826d291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:52.460433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 696/1393] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:52.758762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628f46-c124-4501-9b0a-5c28579a6fc4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:52.759128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628f46-c124-4501-9b0a-5c28579a6fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:52.759493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70628f46-c124-4501-9b0a-5c28579a6fc4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:52.761180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 698/1394] 10.222.0.23 () {54 vars in 961 bytes} [Mon Sep 5 16:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:52.766396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30a88-b386-4460-9942-00be5a3ff985 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:52.766681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30a88-b386-4460-9942-00be5a3ff985 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:52.766948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30a88-b386-4460-9942-00be5a3ff985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:52.846569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-00b30a88-b386-4460-9942-00be5a3ff985 None None] MFA Rules not processed for user `05a016553950460dbb736ced2cae4f62`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:52.847798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-00b30a88-b386-4460-9942-00be5a3ff985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:52.892917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:52.941960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 697/1395] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Sep 5 16:54:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:52.950157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628f46-c124-4501-9b0a-5c28579a6fc4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:52.951688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-70628f46-c124-4501-9b0a-5c28579a6fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:52.966210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:53.061541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70628f46-c124-4501-9b0a-5c28579a6fc4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:53.141723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f906d-5b6c-42f1-a25b-3960490a3094 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05a016553950460dbb736ced2cae4f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:53.142359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f906d-5b6c-42f1-a25b-3960490a3094 service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:53.142661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f906d-5b6c-42f1-a25b-3960490a3094 service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:53.142943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f906d-5b6c-42f1-a25b-3960490a3094 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:53.144552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cf7f906d-5b6c-42f1-a25b-3960490a3094 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:53.215065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:53.258846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f906d-5b6c-42f1-a25b-3960490a3094 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:53.260381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f906d-5b6c-42f1-a25b-3960490a3094 service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:53.278335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 699/1396] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:54.029386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197bd39-a76a-468a-8703-28127af40d79 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:54.029916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197bd39-a76a-468a-8703-28127af40d79 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:54.029916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197bd39-a76a-468a-8703-28127af40d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:54.045323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222b2d5-d2ad-4835-ba5d-0ff6e561d1bf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:54.045608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222b2d5-d2ad-4835-ba5d-0ff6e561d1bf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:54.045879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222b2d5-d2ad-4835-ba5d-0ff6e561d1bf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:54.047251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 700/1397] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:54:54.052383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b069-418e-42fa-a39f-10b657654485 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:54.052668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b069-418e-42fa-a39f-10b657654485 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:54.052942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34b069-418e-42fa-a39f-10b657654485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:54.103975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2197bd39-a76a-468a-8703-28127af40d79 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:54.105205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2197bd39-a76a-468a-8703-28127af40d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:54.126298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ce34b069-418e-42fa-a39f-10b657654485 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:54.127626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ce34b069-418e-42fa-a39f-10b657654485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:54.192429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 698/1398] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:54.228098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 701/1399] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:54:54.235360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db81167-d235-4ab8-9033-b28cb05fcc83 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:54.235648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db81167-d235-4ab8-9033-b28cb05fcc83 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:54.235923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db81167-d235-4ab8-9033-b28cb05fcc83 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:54.237346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 699/1400] 10.222.0.23 () {54 vars in 943 bytes} [Mon Sep 5 16:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:54.243417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:54.243707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:54.244001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:54.273169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:54.332309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] MFA Rules not processed for user `2756036297064cc19d5fd5d1861703c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:54.333509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:54.464347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 702/1401] 10.222.0.23 () {58 vars in 1040 bytes} [Mon Sep 5 16:54:54 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:54:54.468302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db81167-d235-4ab8-9033-b28cb05fcc83 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:54.469840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4db81167-d235-4ab8-9033-b28cb05fcc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:54.518927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:54.520208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:54.538943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:54.598577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:54.616433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db81167-d235-4ab8-9033-b28cb05fcc83 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:54.622024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf286d-ec41-48f6-9c52-b8d2cfc860a7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:54.737656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262d5f9-0495-42d1-ab4e-d96a596d5578 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:54.738179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262d5f9-0495-42d1-ab4e-d96a596d5578 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:54.738455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262d5f9-0495-42d1-ab4e-d96a596d5578 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:54.738723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262d5f9-0495-42d1-ab4e-d96a596d5578 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:54.740393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7262d5f9-0495-42d1-ab4e-d96a596d5578 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:54.746666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390fc51-45ea-43e9-a1a1-dfdc0d1393f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:54.747284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390fc51-45ea-43e9-a1a1-dfdc0d1393f4 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:54.747555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390fc51-45ea-43e9-a1a1-dfdc0d1393f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:54.747844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390fc51-45ea-43e9-a1a1-dfdc0d1393f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:54.749442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2390fc51-45ea-43e9-a1a1-dfdc0d1393f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:54.783459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:54.827773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262d5f9-0495-42d1-ab4e-d96a596d5578 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:54.829367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262d5f9-0495-42d1-ab4e-d96a596d5578 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:54.837323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 700/1402] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 16:54:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 16:54:54.846010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390fc51-45ea-43e9-a1a1-dfdc0d1393f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:54.847290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390fc51-45ea-43e9-a1a1-dfdc0d1393f4 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:54.852436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:54.867574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 703/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:54:56.840481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818e0f3-92b2-43ac-a682-69b9df179ff7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:56.841057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818e0f3-92b2-43ac-a682-69b9df179ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:56.841332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818e0f3-92b2-43ac-a682-69b9df179ff7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:56.842606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 701/1404] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:54:56.847977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22341a3b-d6c8-411c-bade-dcf54d9ea77d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:56.848431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22341a3b-d6c8-411c-bade-dcf54d9ea77d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:56.848630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22341a3b-d6c8-411c-bade-dcf54d9ea77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:56.927773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-22341a3b-d6c8-411c-bade-dcf54d9ea77d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:56.927773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-22341a3b-d6c8-411c-bade-dcf54d9ea77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:57.032991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 704/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:54:57.346919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818e0f3-92b2-43ac-a682-69b9df179ff7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:54:57.352088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4818e0f3-92b2-43ac-a682-69b9df179ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:57.540476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818e0f3-92b2-43ac-a682-69b9df179ff7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:54:57.579945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:57.663719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f1020-cd95-4116-991a-c3fda80de6f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:54:57.664813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1020-cd95-4116-991a-c3fda80de6f6 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:57.665091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1020-cd95-4116-991a-c3fda80de6f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:57.665376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f1020-cd95-4116-991a-c3fda80de6f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:57.666980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-880f1020-cd95-4116-991a-c3fda80de6f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:57.767089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f1020-cd95-4116-991a-c3fda80de6f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:54:57.768876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f1020-cd95-4116-991a-c3fda80de6f6 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:54:57.789837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 702/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:54:59.769832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8e273-116f-4fbb-8776-7c38f0477270 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:59.770830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8e273-116f-4fbb-8776-7c38f0477270 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:59.771201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8e273-116f-4fbb-8776-7c38f0477270 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:59.772935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 705/1407] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:54:59.778342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7b9e1-97c4-4f87-8ddb-db077b9b4aba None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:54:59.778816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7b9e1-97c4-4f87-8ddb-db077b9b4aba None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:54:59.778929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7b9e1-97c4-4f87-8ddb-db077b9b4aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:54:59.836529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:54:59.856605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2be7b9e1-97c4-4f87-8ddb-db077b9b4aba None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:54:59.857897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2be7b9e1-97c4-4f87-8ddb-db077b9b4aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:54:59.972429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 703/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:00.253000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8e273-116f-4fbb-8776-7c38f0477270 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:00.254798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c0a8e273-116f-4fbb-8776-7c38f0477270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:00.339598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:00.418212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8e273-116f-4fbb-8776-7c38f0477270 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:00.543270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304478d-3d4f-47d3-b064-2d6bb5118c34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:00.544230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304478d-3d4f-47d3-b064-2d6bb5118c34 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:00.544508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304478d-3d4f-47d3-b064-2d6bb5118c34 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:00.544777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304478d-3d4f-47d3-b064-2d6bb5118c34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:00.546372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e304478d-3d4f-47d3-b064-2d6bb5118c34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:00.585052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:00.647847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304478d-3d4f-47d3-b064-2d6bb5118c34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:00.649411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304478d-3d4f-47d3-b064-2d6bb5118c34 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:00.668201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 706/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:09.964456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=47 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:10.665665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=31 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:16.428371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dee38-4b57-4d5c-ac69-6232bfc6b419 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:16.428963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dee38-4b57-4d5c-ac69-6232bfc6b419 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:16.428963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0dee38-4b57-4d5c-ac69-6232bfc6b419 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:16.430291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 704/1410] 10.222.0.23 () {54 vars in 918 bytes} [Mon Sep 5 16:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:16.436190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6857f-4789-4ee2-87cb-9ab0da7c529c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:16.436674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6857f-4789-4ee2-87cb-9ab0da7c529c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:16.436854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6857f-4789-4ee2-87cb-9ab0da7c529c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:16.521287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6fc6857f-4789-4ee2-87cb-9ab0da7c529c None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:16.522559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6fc6857f-4789-4ee2-87cb-9ab0da7c529c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:16.640875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 707/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Sep 5 16:55:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:55:16.661688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222860e-5539-48ee-8e0e-36f308d217de None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:16.662034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222860e-5539-48ee-8e0e-36f308d217de None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:16.662303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222860e-5539-48ee-8e0e-36f308d217de None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:16.664207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 705/1412] 10.222.0.23 () {54 vars in 918 bytes} [Mon Sep 5 16:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:16.669857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ce6d-d0ec-41ce-b190-fd866da6a4de None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:16.670150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ce6d-d0ec-41ce-b190-fd866da6a4de None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:16.670450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ce6d-d0ec-41ce-b190-fd866da6a4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:16.747951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:16.751225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-01b3ce6d-d0ec-41ce-b190-fd866da6a4de None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:16.753036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-01b3ce6d-d0ec-41ce-b190-fd866da6a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:16.877484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 708/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Sep 5 16:55:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:55:19.237991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3887f-a2ab-4e24-9f3f-17dbaa90caf0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:19.238589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3887f-a2ab-4e24-9f3f-17dbaa90caf0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:19.238589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3887f-a2ab-4e24-9f3f-17dbaa90caf0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:19.240417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:19.247180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3133e-9b95-4e4a-92c5-e7514f9d4d63 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:19.247604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3133e-9b95-4e4a-92c5-e7514f9d4d63 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:19.247727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3133e-9b95-4e4a-92c5-e7514f9d4d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:19.318784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-78f3133e-9b95-4e4a-92c5-e7514f9d4d63 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:19.319969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-78f3133e-9b95-4e4a-92c5-e7514f9d4d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:19.379888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:19.424879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:19.717444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3887f-a2ab-4e24-9f3f-17dbaa90caf0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:19.718993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-54b3887f-a2ab-4e24-9f3f-17dbaa90caf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:19.869508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3887f-a2ab-4e24-9f3f-17dbaa90caf0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:19.962278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:20.018482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e8251-6ba6-4234-9c98-41039760753b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:20.019265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e8251-6ba6-4234-9c98-41039760753b service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:20.019606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e8251-6ba6-4234-9c98-41039760753b service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:20.019931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e8251-6ba6-4234-9c98-41039760753b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:20.022312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-624e8251-6ba6-4234-9c98-41039760753b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:20.124402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e8251-6ba6-4234-9c98-41039760753b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:20.126000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e8251-6ba6-4234-9c98-41039760753b service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:20.145016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 707/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:21.073198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2141cb-a1f4-46ef-b93b-f2cc7a59e72e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:21.074004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2141cb-a1f4-46ef-b93b-f2cc7a59e72e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:21.074004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2141cb-a1f4-46ef-b93b-f2cc7a59e72e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:21.075716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:21.081649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945db6b8-0f20-41c2-a28c-2063fa4715db None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:21.082308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945db6b8-0f20-41c2-a28c-2063fa4715db None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:21.082547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945db6b8-0f20-41c2-a28c-2063fa4715db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:21.180466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-945db6b8-0f20-41c2-a28c-2063fa4715db None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:21.181861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-945db6b8-0f20-41c2-a28c-2063fa4715db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:21.210040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:21.284811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:21.439168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2141cb-a1f4-46ef-b93b-f2cc7a59e72e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:21.441753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ad2141cb-a1f4-46ef-b93b-f2cc7a59e72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:21.601664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2141cb-a1f4-46ef-b93b-f2cc7a59e72e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:21.643350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:21.717226 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0d891-cb04-4929-9105-93abcda9358e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:21.717851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0d891-cb04-4929-9105-93abcda9358e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:21.718113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0d891-cb04-4929-9105-93abcda9358e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:21.718386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0d891-cb04-4929-9105-93abcda9358e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:21.720028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5cb0d891-cb04-4929-9105-93abcda9358e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:21.812132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0d891-cb04-4929-9105-93abcda9358e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:21.813774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0d891-cb04-4929-9105-93abcda9358e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:21.832455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 711/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:22.717904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211758-8330-48c6-bbb6-0eacb182e34e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:22.718764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211758-8330-48c6-bbb6-0eacb182e34e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:22.718764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211758-8330-48c6-bbb6-0eacb182e34e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:22.720941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:22.728622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e58be-5536-47c9-8cc1-28d1f268794a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:22.729518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e58be-5536-47c9-8cc1-28d1f268794a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:22.729518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e58be-5536-47c9-8cc1-28d1f268794a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:22.787667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:22.813019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-544e58be-5536-47c9-8cc1-28d1f268794a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:22.814247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-544e58be-5536-47c9-8cc1-28d1f268794a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:22.916830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:23.067534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f211758-8330-48c6-bbb6-0eacb182e34e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:23.068889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2f211758-8330-48c6-bbb6-0eacb182e34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:23.240775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f211758-8330-48c6-bbb6-0eacb182e34e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:23.253508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:23.366229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed8d00-1acc-4f4b-a1df-2497f743023b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:23.366834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed8d00-1acc-4f4b-a1df-2497f743023b service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:23.367107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed8d00-1acc-4f4b-a1df-2497f743023b service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:23.367371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed8d00-1acc-4f4b-a1df-2497f743023b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:23.369020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a1ed8d00-1acc-4f4b-a1df-2497f743023b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:23.477221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed8d00-1acc-4f4b-a1df-2497f743023b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:23.479760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed8d00-1acc-4f4b-a1df-2497f743023b service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:23.499302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 710/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:24.365556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d018-eba1-4f07-afe4-fc47dfe8fe23 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:24.366421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d018-eba1-4f07-afe4-fc47dfe8fe23 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:24.366421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d018-eba1-4f07-afe4-fc47dfe8fe23 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:24.368103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:24.374152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75310f-112b-4840-b18f-f55ad5ae32ca None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:24.374601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75310f-112b-4840-b18f-f55ad5ae32ca None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:24.374818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75310f-112b-4840-b18f-f55ad5ae32ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:24.396047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:24.455124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6d75310f-112b-4840-b18f-f55ad5ae32ca None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:24.456400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6d75310f-112b-4840-b18f-f55ad5ae32ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:24.560186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:24.713539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d018-eba1-4f07-afe4-fc47dfe8fe23 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:24.714952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4d40d018-eba1-4f07-afe4-fc47dfe8fe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:24.833716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:24.872419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d018-eba1-4f07-afe4-fc47dfe8fe23 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:24.998766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e8118-994a-400e-8286-9553ce8e87e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:24.999295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e8118-994a-400e-8286-9553ce8e87e0 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:24.999576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e8118-994a-400e-8286-9553ce8e87e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:24.999833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e8118-994a-400e-8286-9553ce8e87e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:25.001480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-455e8118-994a-400e-8286-9553ce8e87e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:25.088008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:25.113900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e8118-994a-400e-8286-9553ce8e87e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:25.116482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e8118-994a-400e-8286-9553ce8e87e0 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:25.142287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 714/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:26.012519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f003-1e79-48b0-927f-5ae310b481ce None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:26.013371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f003-1e79-48b0-927f-5ae310b481ce None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:26.013371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f003-1e79-48b0-927f-5ae310b481ce None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:26.015512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:26.021565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c62a7-0fd8-42e4-a845-7bc393599e28 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:26.022312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c62a7-0fd8-42e4-a845-7bc393599e28 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:26.022733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c62a7-0fd8-42e4-a845-7bc393599e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:26.103611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-af1c62a7-0fd8-42e4-a845-7bc393599e28 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:26.105184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af1c62a7-0fd8-42e4-a845-7bc393599e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:26.225393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:26.377856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f003-1e79-48b0-927f-5ae310b481ce None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:26.380201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-88a5f003-1e79-48b0-927f-5ae310b481ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:26.472281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:26.539046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f003-1e79-48b0-927f-5ae310b481ce None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:26.658664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d21c37-d2e6-4741-98d5-6f8a845c0f72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:26.659294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d21c37-d2e6-4741-98d5-6f8a845c0f72 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:26.659571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d21c37-d2e6-4741-98d5-6f8a845c0f72 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:26.659841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d21c37-d2e6-4741-98d5-6f8a845c0f72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:26.661498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-40d21c37-d2e6-4741-98d5-6f8a845c0f72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:26.716846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:26.770579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d21c37-d2e6-4741-98d5-6f8a845c0f72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:26.772195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d21c37-d2e6-4741-98d5-6f8a845c0f72 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:26.797654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 713/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:55:27.609358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abe161-7af6-4fac-8bb6-e7085f299883 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:27.610313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abe161-7af6-4fac-8bb6-e7085f299883 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:27.610313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abe161-7af6-4fac-8bb6-e7085f299883 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:27.612413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:27.618486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07564e2d-d841-4de6-80c8-941a1aa66d63 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:27.619051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07564e2d-d841-4de6-80c8-941a1aa66d63 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:27.619331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07564e2d-d841-4de6-80c8-941a1aa66d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:27.700221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-07564e2d-d841-4de6-80c8-941a1aa66d63 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:27.701348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-07564e2d-d841-4de6-80c8-941a1aa66d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:27.808061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:27.961090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abe161-7af6-4fac-8bb6-e7085f299883 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:27.963427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e8abe161-7af6-4fac-8bb6-e7085f299883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:28.021836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:28.131140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abe161-7af6-4fac-8bb6-e7085f299883 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:28.264908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd95bdd-c24a-45d2-a445-be2c042620c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:28.265457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd95bdd-c24a-45d2-a445-be2c042620c9 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:28.265740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd95bdd-c24a-45d2-a445-be2c042620c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:28.266018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd95bdd-c24a-45d2-a445-be2c042620c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:28.267650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dfd95bdd-c24a-45d2-a445-be2c042620c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:28.274998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:28.366990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd95bdd-c24a-45d2-a445-be2c042620c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:28.368719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd95bdd-c24a-45d2-a445-be2c042620c9 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:28.388044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 717/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:29.185069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97f5-ff4a-40e7-b8fd-12e7bdc3cd6b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:29.186036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97f5-ff4a-40e7-b8fd-12e7bdc3cd6b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:29.186036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef97f5-ff4a-40e7-b8fd-12e7bdc3cd6b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:29.188232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:29.193404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d53976-2ae3-4224-b4ff-051f4a2c2a2c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:29.193862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d53976-2ae3-4224-b4ff-051f4a2c2a2c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:29.193965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d53976-2ae3-4224-b4ff-051f4a2c2a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:29.274793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-11d53976-2ae3-4224-b4ff-051f4a2c2a2c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:29.276017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-11d53976-2ae3-4224-b4ff-051f4a2c2a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:29.344932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:29.375904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:29.528924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef97f5-ff4a-40e7-b8fd-12e7bdc3cd6b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:29.530328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bfef97f5-ff4a-40e7-b8fd-12e7bdc3cd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:29.551628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:29.691036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef97f5-ff4a-40e7-b8fd-12e7bdc3cd6b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:29.784601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:29.816952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932e5ce-5418-4e84-ba2f-c54e40c2b11f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:29.817599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932e5ce-5418-4e84-ba2f-c54e40c2b11f service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:29.817895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932e5ce-5418-4e84-ba2f-c54e40c2b11f service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:29.818162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932e5ce-5418-4e84-ba2f-c54e40c2b11f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:29.819786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2932e5ce-5418-4e84-ba2f-c54e40c2b11f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:29.927972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932e5ce-5418-4e84-ba2f-c54e40c2b11f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:29.930416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932e5ce-5418-4e84-ba2f-c54e40c2b11f service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:29.949823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 716/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:30.749235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8734144-dc85-4de3-bd7d-cc7b6d7762b8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:30.750110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8734144-dc85-4de3-bd7d-cc7b6d7762b8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:30.750110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8734144-dc85-4de3-bd7d-cc7b6d7762b8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:30.752204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:30.760286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0526f-1a0f-43b2-9df6-26af023c3303 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:30.760734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0526f-1a0f-43b2-9df6-26af023c3303 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:30.760864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0526f-1a0f-43b2-9df6-26af023c3303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:30.852033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4dd0526f-1a0f-43b2-9df6-26af023c3303 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:30.853784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4dd0526f-1a0f-43b2-9df6-26af023c3303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:30.899511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:30.962742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:31.119383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8734144-dc85-4de3-bd7d-cc7b6d7762b8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:31.120971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f8734144-dc85-4de3-bd7d-cc7b6d7762b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:31.276938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8734144-dc85-4de3-bd7d-cc7b6d7762b8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:31.333953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:31.397119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d42080-a196-4b5f-854e-fcf8282b46ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:31.397900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d42080-a196-4b5f-854e-fcf8282b46ff service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:31.398267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d42080-a196-4b5f-854e-fcf8282b46ff service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:31.398645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d42080-a196-4b5f-854e-fcf8282b46ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:31.401188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c1d42080-a196-4b5f-854e-fcf8282b46ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:31.494330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d42080-a196-4b5f-854e-fcf8282b46ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:31.495940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d42080-a196-4b5f-854e-fcf8282b46ff service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:31.515306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 720/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:32.316210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7697-d177-4a85-82ba-60c1e6eda396 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:32.316759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7697-d177-4a85-82ba-60c1e6eda396 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:32.316759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7697-d177-4a85-82ba-60c1e6eda396 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:32.318834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:55:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:32.324069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddbbfb-028c-4e69-9e57-2022741dae98 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:32.324545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddbbfb-028c-4e69-9e57-2022741dae98 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:32.324662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddbbfb-028c-4e69-9e57-2022741dae98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:32.406642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fbddbbfb-028c-4e69-9e57-2022741dae98 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:32.407898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fbddbbfb-028c-4e69-9e57-2022741dae98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:32.410328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:32.516352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:32.684067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7697-d177-4a85-82ba-60c1e6eda396 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:32.686278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c74b7697-d177-4a85-82ba-60c1e6eda396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:32.855081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7697-d177-4a85-82ba-60c1e6eda396 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:32.883225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:32.982598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf3899-b850-46a2-a839-9b00eae890c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:32.983136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3899-b850-46a2-a839-9b00eae890c0 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:32.983405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3899-b850-46a2-a839-9b00eae890c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:32.983675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf3899-b850-46a2-a839-9b00eae890c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:32.985311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e2cf3899-b850-46a2-a839-9b00eae890c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:33.086290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf3899-b850-46a2-a839-9b00eae890c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:33.087961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cf3899-b850-46a2-a839-9b00eae890c0 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:33.107526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 719/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:34.938770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d584e7d-6932-485f-9835-1c1bc03327b4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:34.939367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d584e7d-6932-485f-9835-1c1bc03327b4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:34.939367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d584e7d-6932-485f-9835-1c1bc03327b4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:34.940773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:34.947630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284b6df-5c6a-442f-9c39-66fc4d77a8c8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:34.948424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284b6df-5c6a-442f-9c39-66fc4d77a8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:34.948638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284b6df-5c6a-442f-9c39-66fc4d77a8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:34.997814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:35.023097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5284b6df-5c6a-442f-9c39-66fc4d77a8c8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:35.024324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5284b6df-5c6a-442f-9c39-66fc4d77a8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:35.127613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:35.424593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d584e7d-6932-485f-9835-1c1bc03327b4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:35.426205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9d584e7d-6932-485f-9835-1c1bc03327b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:35.559561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d584e7d-6932-485f-9835-1c1bc03327b4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:35.598170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:35.644771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fda02-5482-4305-beee-d87d1531ce16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:35.645288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fda02-5482-4305-beee-d87d1531ce16 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:35.645566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fda02-5482-4305-beee-d87d1531ce16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:35.645833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fda02-5482-4305-beee-d87d1531ce16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:35.647447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-664fda02-5482-4305-beee-d87d1531ce16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:35.784968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fda02-5482-4305-beee-d87d1531ce16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:35.786457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fda02-5482-4305-beee-d87d1531ce16 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:35.806780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 723/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:55:37.894330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0ea1f-63fe-419f-9f65-1eb5d6f896a3 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:37.894926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0ea1f-63fe-419f-9f65-1eb5d6f896a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:37.894926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0ea1f-63fe-419f-9f65-1eb5d6f896a3 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:37.896205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:37.902590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08fbf4-8adf-4a68-981f-9eb7778cc022 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:37.903256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08fbf4-8adf-4a68-981f-9eb7778cc022 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:37.903416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08fbf4-8adf-4a68-981f-9eb7778cc022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:37.987567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0f08fbf4-8adf-4a68-981f-9eb7778cc022 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:37.989380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0f08fbf4-8adf-4a68-981f-9eb7778cc022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:38.004776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:38.118456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:38.409490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0ea1f-63fe-419f-9f65-1eb5d6f896a3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:38.410966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-00d0ea1f-63fe-419f-9f65-1eb5d6f896a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:38.527997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0ea1f-63fe-419f-9f65-1eb5d6f896a3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:38.556643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:38.643568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca39c43-8e78-48d8-ba3c-a82d9d35600f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:38.644226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39c43-8e78-48d8-ba3c-a82d9d35600f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:38.644505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39c43-8e78-48d8-ba3c-a82d9d35600f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:38.644767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca39c43-8e78-48d8-ba3c-a82d9d35600f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:38.647189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7ca39c43-8e78-48d8-ba3c-a82d9d35600f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:38.777329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca39c43-8e78-48d8-ba3c-a82d9d35600f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:38.778765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca39c43-8e78-48d8-ba3c-a82d9d35600f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:38.801675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 722/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:55:41.016213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752807-219b-43c6-9d1a-34c4894c8885 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:41.016798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752807-219b-43c6-9d1a-34c4894c8885 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:41.016798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65752807-219b-43c6-9d1a-34c4894c8885 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:41.018695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:41.024296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227e349-c016-44da-a77c-bc8169ad0cfd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:41.025231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227e349-c016-44da-a77c-bc8169ad0cfd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:41.025517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227e349-c016-44da-a77c-bc8169ad0cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:41.079778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:41.101280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5227e349-c016-44da-a77c-bc8169ad0cfd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:41.102539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5227e349-c016-44da-a77c-bc8169ad0cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:41.204142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:41.490997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65752807-219b-43c6-9d1a-34c4894c8885 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:41.492582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65752807-219b-43c6-9d1a-34c4894c8885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:41.605920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65752807-219b-43c6-9d1a-34c4894c8885 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:41.649079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:41.682925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76118c3b-6c16-467e-9d36-b545c5763393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:41.683537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76118c3b-6c16-467e-9d36-b545c5763393 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:41.683817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76118c3b-6c16-467e-9d36-b545c5763393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:41.684086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76118c3b-6c16-467e-9d36-b545c5763393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:41.685690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-76118c3b-6c16-467e-9d36-b545c5763393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:41.808079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76118c3b-6c16-467e-9d36-b545c5763393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:41.809574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76118c3b-6c16-467e-9d36-b545c5763393 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:41.828432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 726/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:43.849771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17da97-bfbc-4cca-8d73-9cc7ff8050f9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:43.850734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17da97-bfbc-4cca-8d73-9cc7ff8050f9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:43.850734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17da97-bfbc-4cca-8d73-9cc7ff8050f9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:43.852841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:43.858899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acfe8f-fbb3-4b65-ad85-70cf282cb980 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:43.859371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acfe8f-fbb3-4b65-ad85-70cf282cb980 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:43.859467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acfe8f-fbb3-4b65-ad85-70cf282cb980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:43.928645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c9acfe8f-fbb3-4b65-ad85-70cf282cb980 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:43.929834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c9acfe8f-fbb3-4b65-ad85-70cf282cb980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:43.951873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:44.033747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:44.321015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17da97-bfbc-4cca-8d73-9cc7ff8050f9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:44.322636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ff17da97-bfbc-4cca-8d73-9cc7ff8050f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:44.434480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17da97-bfbc-4cca-8d73-9cc7ff8050f9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:44.455642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:44.509443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756be2cb-1acb-404c-b46c-300cb21d12df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:44.510077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756be2cb-1acb-404c-b46c-300cb21d12df service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:44.510354 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756be2cb-1acb-404c-b46c-300cb21d12df service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:44.510619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756be2cb-1acb-404c-b46c-300cb21d12df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:44.512256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-756be2cb-1acb-404c-b46c-300cb21d12df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:44.627175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756be2cb-1acb-404c-b46c-300cb21d12df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:44.628556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756be2cb-1acb-404c-b46c-300cb21d12df service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:44.648710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 725/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:55:46.677051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087ff39-634f-4a7e-a884-da59ca6eb33b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:46.677630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087ff39-634f-4a7e-a884-da59ca6eb33b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:46.677630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087ff39-634f-4a7e-a884-da59ca6eb33b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:46.678886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:46.685521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2005c4-b6fd-41ad-88e8-1e2989d7a500 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:46.686320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2005c4-b6fd-41ad-88e8-1e2989d7a500 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:46.686446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2005c4-b6fd-41ad-88e8-1e2989d7a500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:46.765871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:46.769693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bf2005c4-b6fd-41ad-88e8-1e2989d7a500 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:46.771355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf2005c4-b6fd-41ad-88e8-1e2989d7a500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:46.873811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:47.153533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087ff39-634f-4a7e-a884-da59ca6eb33b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:47.155080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6087ff39-634f-4a7e-a884-da59ca6eb33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:47.271838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087ff39-634f-4a7e-a884-da59ca6eb33b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:47.332600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:47.358653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38942b1-89b5-405f-92cb-9f2496e6879e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:47.359260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38942b1-89b5-405f-92cb-9f2496e6879e service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:47.359536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38942b1-89b5-405f-92cb-9f2496e6879e service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:47.359804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38942b1-89b5-405f-92cb-9f2496e6879e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:47.361424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c38942b1-89b5-405f-92cb-9f2496e6879e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:47.479326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38942b1-89b5-405f-92cb-9f2496e6879e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:47.480780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38942b1-89b5-405f-92cb-9f2496e6879e service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:47.500074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 729/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:49.491379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6dee-acde-4039-9e24-0dbec5b14271 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:49.492004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6dee-acde-4039-9e24-0dbec5b14271 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:49.492004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6dee-acde-4039-9e24-0dbec5b14271 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:49.493388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:49.499784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62c296-5d58-4608-b1bd-ab2c3929d1c2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:49.500280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62c296-5d58-4608-b1bd-ab2c3929d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:49.500397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62c296-5d58-4608-b1bd-ab2c3929d1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:49.584165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4d62c296-5d58-4608-b1bd-ab2c3929d1c2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:49.585422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4d62c296-5d58-4608-b1bd-ab2c3929d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:49.617515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:49.689888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:49.973074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb6dee-acde-4039-9e24-0dbec5b14271 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:49.974638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0dfb6dee-acde-4039-9e24-0dbec5b14271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:50.088818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb6dee-acde-4039-9e24-0dbec5b14271 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:50.119849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:50.167226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b5fa3-4bc7-43c6-bd8e-859e34d7037c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:50.167832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b5fa3-4bc7-43c6-bd8e-859e34d7037c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:50.168104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b5fa3-4bc7-43c6-bd8e-859e34d7037c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:50.168428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b5fa3-4bc7-43c6-bd8e-859e34d7037c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:50.169996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-894b5fa3-4bc7-43c6-bd8e-859e34d7037c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:50.286593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b5fa3-4bc7-43c6-bd8e-859e34d7037c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:50.287942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b5fa3-4bc7-43c6-bd8e-859e34d7037c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:50.307185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 728/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:55:52.272471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f09c8-0171-4400-8c20-8f2a88a0e079 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:52.273028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f09c8-0171-4400-8c20-8f2a88a0e079 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:52.273028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f09c8-0171-4400-8c20-8f2a88a0e079 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:52.274381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:52.280905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e34f-23d7-4277-b3cc-3684df3e82ec None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:52.281598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e34f-23d7-4277-b3cc-3684df3e82ec None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:52.281785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24e34f-23d7-4277-b3cc-3684df3e82ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:52.358822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5c24e34f-23d7-4277-b3cc-3684df3e82ec None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:52.360022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5c24e34f-23d7-4277-b3cc-3684df3e82ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:52.369060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:52.464778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:55:52.747270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f09c8-0171-4400-8c20-8f2a88a0e079 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:52.749821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a6f09c8-0171-4400-8c20-8f2a88a0e079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:52.862360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f09c8-0171-4400-8c20-8f2a88a0e079 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:52.918569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:52.937554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810fd26-90f0-430d-af94-0746ee2831af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:52.938156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810fd26-90f0-430d-af94-0746ee2831af service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:52.938431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810fd26-90f0-430d-af94-0746ee2831af service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:52.938704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810fd26-90f0-430d-af94-0746ee2831af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:52.940301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4810fd26-90f0-430d-af94-0746ee2831af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:53.068981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810fd26-90f0-430d-af94-0746ee2831af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:53.070361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810fd26-90f0-430d-af94-0746ee2831af service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:53.089468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 732/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:55.034328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fbc40-4e45-471c-bed5-cdc8fd90d5f4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:55.035244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fbc40-4e45-471c-bed5-cdc8fd90d5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:55.035244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fbc40-4e45-471c-bed5-cdc8fd90d5f4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:55.037508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:55.044254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3262-0999-4ee0-a234-97b61b623124 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:55.044720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3262-0999-4ee0-a234-97b61b623124 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:55.044811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3262-0999-4ee0-a234-97b61b623124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:55.119085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8dda3262-0999-4ee0-a234-97b61b623124 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:55.120353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8dda3262-0999-4ee0-a234-97b61b623124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:55.160213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:55.221565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:55:55.518854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fbc40-4e45-471c-bed5-cdc8fd90d5f4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:55.521056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1d6fbc40-4e45-471c-bed5-cdc8fd90d5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:55.642103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fbc40-4e45-471c-bed5-cdc8fd90d5f4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:55.682790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:55.721944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ab686-8ed0-442b-9432-53a57e982287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:55.722572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ab686-8ed0-442b-9432-53a57e982287 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:55.722860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ab686-8ed0-442b-9432-53a57e982287 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:55.723125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ab686-8ed0-442b-9432-53a57e982287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:55.724751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dd9ab686-8ed0-442b-9432-53a57e982287 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:55.847957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ab686-8ed0-442b-9432-53a57e982287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:55.849428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ab686-8ed0-442b-9432-53a57e982287 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:55.869761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 731/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:55:57.801384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076297ec-8bcf-4b69-8631-7765a75bd551 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:57.801906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076297ec-8bcf-4b69-8631-7765a75bd551 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:57.801906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076297ec-8bcf-4b69-8631-7765a75bd551 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:57.803932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:55:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:55:57.812207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9747f7-d033-444f-96ed-8b5df03b15a8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:57.812700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9747f7-d033-444f-96ed-8b5df03b15a8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:57.812806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9747f7-d033-444f-96ed-8b5df03b15a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:57.881603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fe9747f7-d033-444f-96ed-8b5df03b15a8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:55:57.882792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fe9747f7-d033-444f-96ed-8b5df03b15a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:57.901842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:57.982670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:55:58.263301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076297ec-8bcf-4b69-8631-7765a75bd551 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:55:58.264874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-076297ec-8bcf-4b69-8631-7765a75bd551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:58.381693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076297ec-8bcf-4b69-8631-7765a75bd551 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:55:58.453872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:55:58.463396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d0d2f-5833-488b-8c3a-acfe631decb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:55:58.464313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0d2f-5833-488b-8c3a-acfe631decb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:55:58.464704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0d2f-5833-488b-8c3a-acfe631decb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:55:58.465072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0d2f-5833-488b-8c3a-acfe631decb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:55:58.467533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-476d0d2f-5833-488b-8c3a-acfe631decb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:55:58.590112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d0d2f-5833-488b-8c3a-acfe631decb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:55:58.591561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d0d2f-5833-488b-8c3a-acfe631decb6 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:55:58.610452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 735/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:56:00.505396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f1829-a00d-4765-a85c-b3e647bb4378 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:00.506042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f1829-a00d-4765-a85c-b3e647bb4378 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:00.506042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f1829-a00d-4765-a85c-b3e647bb4378 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:00.507258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:00.513887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2ac76-8961-4575-94f6-0472d36bcd1d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:00.514658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2ac76-8961-4575-94f6-0472d36bcd1d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:00.514840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2ac76-8961-4575-94f6-0472d36bcd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:00.587186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c9c2ac76-8961-4575-94f6-0472d36bcd1d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:00.588445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c9c2ac76-8961-4575-94f6-0472d36bcd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:00.643697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:00.700423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:00.981756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f1829-a00d-4765-a85c-b3e647bb4378 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:00.984248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cd5f1829-a00d-4765-a85c-b3e647bb4378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:01.104734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f1829-a00d-4765-a85c-b3e647bb4378 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:01.154666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:01.191757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f6cd9-f01a-4b63-91e7-8863756795b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:01.192384 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6cd9-f01a-4b63-91e7-8863756795b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:01.192664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6cd9-f01a-4b63-91e7-8863756795b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:01.192954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f6cd9-f01a-4b63-91e7-8863756795b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:01.194562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-510f6cd9-f01a-4b63-91e7-8863756795b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:01.318350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f6cd9-f01a-4b63-91e7-8863756795b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:01.319705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f6cd9-f01a-4b63-91e7-8863756795b3 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:01.339556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 734/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:56:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:56:10.698154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=22 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:11.336710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=66 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:20.757974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cd8b4-0466-41ea-9104-19185d5da9d7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:20.757974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cd8b4-0466-41ea-9104-19185d5da9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:20.759039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cd8b4-0466-41ea-9104-19185d5da9d7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:20.759982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 737/1471] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:20.765515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298438-b781-4d0d-9abf-3647579919f2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:20.765969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298438-b781-4d0d-9abf-3647579919f2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:20.765969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298438-b781-4d0d-9abf-3647579919f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:20.835393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-35298438-b781-4d0d-9abf-3647579919f2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:20.836708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-35298438-b781-4d0d-9abf-3647579919f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:20.945184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 735/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:21.259572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e914-68ab-49d1-9119-5b324b043ec8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:21.259872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e914-68ab-49d1-9119-5b324b043ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:21.260122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97e914-68ab-49d1-9119-5b324b043ec8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:21.261524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 738/1473] 10.222.0.23 () {54 vars in 961 bytes} [Mon Sep 5 16:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:21.266668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de1a80-a7ad-4ca4-bbb2-4c8aa6b013fb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:21.266954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de1a80-a7ad-4ca4-bbb2-4c8aa6b013fb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:21.267230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de1a80-a7ad-4ca4-bbb2-4c8aa6b013fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:21.361453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-58de1a80-a7ad-4ca4-bbb2-4c8aa6b013fb None None] MFA Rules not processed for user `05a016553950460dbb736ced2cae4f62`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:21.362645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58de1a80-a7ad-4ca4-bbb2-4c8aa6b013fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:21.386872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:21.448331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 736/1474] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Sep 5 16:56:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:21.456360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97e914-68ab-49d1-9119-5b324b043ec8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:21.457929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bc97e914-68ab-49d1-9119-5b324b043ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:21.570083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97e914-68ab-49d1-9119-5b324b043ec8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:21.650216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeacb86-c9bd-4565-86ed-9a6130afe1f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05a016553950460dbb736ced2cae4f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:21.650848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeacb86-c9bd-4565-86ed-9a6130afe1f5 service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:21.651105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeacb86-c9bd-4565-86ed-9a6130afe1f5 service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:21.651398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeacb86-c9bd-4565-86ed-9a6130afe1f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:21.652994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ceeacb86-c9bd-4565-86ed-9a6130afe1f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:21.711310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:21.777938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeacb86-c9bd-4565-86ed-9a6130afe1f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:21.779417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeacb86-c9bd-4565-86ed-9a6130afe1f5 service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:21.804184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 739/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:56:23.811001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce6bb0-2fc4-4e90-a966-736f3cecf753 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:23.811616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce6bb0-2fc4-4e90-a966-736f3cecf753 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:23.811616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce6bb0-2fc4-4e90-a966-736f3cecf753 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:23.813469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 737/1476] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:23.819674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33a557-e4d1-4974-b5b0-14e2c806e881 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:23.820115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33a557-e4d1-4974-b5b0-14e2c806e881 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:23.820249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33a557-e4d1-4974-b5b0-14e2c806e881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:23.910058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0e33a557-e4d1-4974-b5b0-14e2c806e881 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:23.911280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0e33a557-e4d1-4974-b5b0-14e2c806e881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:24.012584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 740/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:24.300856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce6bb0-2fc4-4e90-a966-736f3cecf753 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:24.303346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c5ce6bb0-2fc4-4e90-a966-736f3cecf753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:24.441981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce6bb0-2fc4-4e90-a966-736f3cecf753 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:24.499530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:24.548714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d1ae1-a61b-45e8-9852-4e383975ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:24.549329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d1ae1-a61b-45e8-9852-4e383975ee06 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:24.549603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d1ae1-a61b-45e8-9852-4e383975ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:24.549867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d1ae1-a61b-45e8-9852-4e383975ee06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:24.550502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d1ae1-a61b-45e8-9852-4e383975ee06 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:24.552257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-c53d1ae1-a61b-45e8-9852-4e383975ee06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:56:24.552257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 16:56:24.552257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 16:56:24.553438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d1ae1-a61b-45e8-9852-4e383975ee06 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:24.573944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 738/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Sep 5 16:56:24 2022] GET /identity/v3/projects => generated 2553 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:56:26.488094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4651-5512-4646-899f-6ec58693a5a7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:26.488658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4651-5512-4646-899f-6ec58693a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:26.488658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4651-5512-4646-899f-6ec58693a5a7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:26.490660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 741/1479] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:56:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:26.499078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ed277-ace8-4e5e-b621-26413e43fd48 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:26.499512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ed277-ace8-4e5e-b621-26413e43fd48 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:26.499639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ed277-ace8-4e5e-b621-26413e43fd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:26.577197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b81ed277-ace8-4e5e-b621-26413e43fd48 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:26.578392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b81ed277-ace8-4e5e-b621-26413e43fd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:26.680911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 739/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:26.988222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c4651-5512-4646-899f-6ec58693a5a7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:26.989801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3a5c4651-5512-4646-899f-6ec58693a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:27.000012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:27.161800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c4651-5512-4646-899f-6ec58693a5a7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:27.271374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:27.311362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c80263-bb78-4127-8cf8-35af318aa244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:27.311988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c80263-bb78-4127-8cf8-35af318aa244 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:27.312297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c80263-bb78-4127-8cf8-35af318aa244 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:27.312574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c80263-bb78-4127-8cf8-35af318aa244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:27.314174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-52c80263-bb78-4127-8cf8-35af318aa244 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:27.413651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c80263-bb78-4127-8cf8-35af318aa244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:27.415470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c80263-bb78-4127-8cf8-35af318aa244 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:27.435058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 742/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:56:29.546581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f1d74-59ad-495c-8ed5-1dd997f2eb25 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:29.547123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f1d74-59ad-495c-8ed5-1dd997f2eb25 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:29.547123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f1d74-59ad-495c-8ed5-1dd997f2eb25 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:29.548457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 740/1482] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:29.553900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035e24f-d07e-4e26-aee8-2de23aa42950 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:29.554371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035e24f-d07e-4e26-aee8-2de23aa42950 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:29.554572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035e24f-d07e-4e26-aee8-2de23aa42950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:29.623984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5035e24f-d07e-4e26-aee8-2de23aa42950 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:29.625542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5035e24f-d07e-4e26-aee8-2de23aa42950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:29.799821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:29.856653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 743/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:30.146555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f1d74-59ad-495c-8ed5-1dd997f2eb25 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:30.149095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-359f1d74-59ad-495c-8ed5-1dd997f2eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:30.180300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:30.266670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f1d74-59ad-495c-8ed5-1dd997f2eb25 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:30.345071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89a615-ccbc-4ed7-9459-8cf13b9466f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:30.345672 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89a615-ccbc-4ed7-9459-8cf13b9466f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:30.345943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89a615-ccbc-4ed7-9459-8cf13b9466f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:30.346204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89a615-ccbc-4ed7-9459-8cf13b9466f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:30.347757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b89a615-ccbc-4ed7-9459-8cf13b9466f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:30.424550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:30.461503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89a615-ccbc-4ed7-9459-8cf13b9466f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:30.462948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b89a615-ccbc-4ed7-9459-8cf13b9466f4 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:30.482271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 741/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:56:32.457135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5e258-aed2-4159-bc2c-245f6661937b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:32.457723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5e258-aed2-4159-bc2c-245f6661937b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:32.457723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5e258-aed2-4159-bc2c-245f6661937b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:32.459078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 744/1485] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:32.464943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9aadd-b801-4e44-814c-d58184b25f1f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:32.465387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9aadd-b801-4e44-814c-d58184b25f1f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:32.465502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9aadd-b801-4e44-814c-d58184b25f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:32.535717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-13d9aadd-b801-4e44-814c-d58184b25f1f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:32.536980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-13d9aadd-b801-4e44-814c-d58184b25f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:32.642454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 742/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:32.944655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5e258-aed2-4159-bc2c-245f6661937b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:32.946936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c9c5e258-aed2-4159-bc2c-245f6661937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:33.056348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5e258-aed2-4159-bc2c-245f6661937b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:33.129434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:33.131554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae609342-63a5-42e4-ab08-80b2745ccbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:33.132262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609342-63a5-42e4-ab08-80b2745ccbce service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:33.132693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609342-63a5-42e4-ab08-80b2745ccbce service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:33.133042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae609342-63a5-42e4-ab08-80b2745ccbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:33.135366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ae609342-63a5-42e4-ab08-80b2745ccbce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:33.253878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae609342-63a5-42e4-ab08-80b2745ccbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:33.255238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae609342-63a5-42e4-ab08-80b2745ccbce service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:33.274053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 745/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:56:35.326757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9979f2-29f9-4031-9fbf-a3c9eb588c69 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:35.327395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9979f2-29f9-4031-9fbf-a3c9eb588c69 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:35.327395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9979f2-29f9-4031-9fbf-a3c9eb588c69 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:35.328661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 743/1488] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:35.335034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311420-6797-45be-b82d-156b659e995c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:35.335763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311420-6797-45be-b82d-156b659e995c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:35.335918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d311420-6797-45be-b82d-156b659e995c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:35.410942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4d311420-6797-45be-b82d-156b659e995c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:35.412173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4d311420-6797-45be-b82d-156b659e995c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:35.470855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:35.523803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 746/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:35.824062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9979f2-29f9-4031-9fbf-a3c9eb588c69 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:35.825648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9e9979f2-29f9-4031-9fbf-a3c9eb588c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:35.872645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:35.947763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9979f2-29f9-4031-9fbf-a3c9eb588c69 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:36.033188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71832ded-efdd-4918-a2cd-2ebda72392ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:36.033807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832ded-efdd-4918-a2cd-2ebda72392ab service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:36.034088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832ded-efdd-4918-a2cd-2ebda72392ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:36.034357 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71832ded-efdd-4918-a2cd-2ebda72392ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:36.035948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-71832ded-efdd-4918-a2cd-2ebda72392ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:36.128189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:36.160614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71832ded-efdd-4918-a2cd-2ebda72392ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:36.162049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71832ded-efdd-4918-a2cd-2ebda72392ab service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:36.180524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 744/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:56:38.332915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9f57-4010-4516-9a3c-62ef09cd714e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:38.333499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9f57-4010-4516-9a3c-62ef09cd714e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:38.333499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9f57-4010-4516-9a3c-62ef09cd714e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:38.335485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 747/1491] 10.222.0.23 () {54 vars in 927 bytes} [Mon Sep 5 16:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:56:38.344356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d26e4-c37a-44d6-b0b6-9b49d8f82f12 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:38.344784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d26e4-c37a-44d6-b0b6-9b49d8f82f12 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:38.344924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d26e4-c37a-44d6-b0b6-9b49d8f82f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:38.418495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b32d26e4-c37a-44d6-b0b6-9b49d8f82f12 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:56:38.419671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b32d26e4-c37a-44d6-b0b6-9b49d8f82f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:38.542689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 745/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Mon Sep 5 16:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:56:38.832632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe9f57-4010-4516-9a3c-62ef09cd714e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:56:38.834985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5cbe9f57-4010-4516-9a3c-62ef09cd714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:38.958065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe9f57-4010-4516-9a3c-62ef09cd714e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:56:39.042497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d856851-7249-4db5-ac2a-9fc9f0a55a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:56:39.043117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d856851-7249-4db5-ac2a-9fc9f0a55a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:56:39.043392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d856851-7249-4db5-ac2a-9fc9f0a55a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:56:39.043658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d856851-7249-4db5-ac2a-9fc9f0a55a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:56:39.045294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1d856851-7249-4db5-ac2a-9fc9f0a55a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:56:39.055437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:39.165465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d856851-7249-4db5-ac2a-9fc9f0a55a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:56:39.166926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d856851-7249-4db5-ac2a-9fc9f0a55a3d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:56:39.186175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 748/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:56:48.540588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:56:49.183810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=50 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:53.711189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:53.712061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:53.712061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:53.803687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:57:53.804924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:53.907819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 746/1494] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 16:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:57:53.918282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09deba1-ae17-4e03-9107-a1a52824046c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:53.918809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09deba1-ae17-4e03-9107-a1a52824046c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:53.918906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09deba1-ae17-4e03-9107-a1a52824046c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:54.008628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b09deba1-ae17-4e03-9107-a1a52824046c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:57:54.009750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b09deba1-ae17-4e03-9107-a1a52824046c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:54.116718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 749/1495] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 16:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:57:54.121836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:54.123055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:54.135426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b710d21-0b04-451a-a533-abbbe303e6e1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:54.205900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:54.241803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:54.242413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:54.242686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:54.242949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:54.243591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 16:57:54.243891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:54.244974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:54.244974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 16:57:54.244974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 16:57:54.245874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:54.258130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 747/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 16:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:57:54.267168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09deba1-ae17-4e03-9107-a1a52824046c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:54.276856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09deba1-ae17-4e03-9107-a1a52824046c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:54.341190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:54.378691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:54.379310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:54.379586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:54.379848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:54.380520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:54.382494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:54.382494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Sep 05 16:57:54.382494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: Sep 05 16:57:54.383736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:54.444934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 750/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 16:57:54 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:57:54.448655 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:54.450282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:54.485995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638e85-6329-4c89-a51c-9860123dd2f9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:54.585886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:54.586519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:54.586795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:54.587064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:54.587688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:54.588737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:54.588737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 16:57:54.588737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 16:57:54.589959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:54.642784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 748/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 16:57:54 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 16:57:54.647684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:54.650033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:54.718891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10895b-57ef-44db-930e-10f81cec33ec admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:54.813434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:54.825332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:54.825950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:54.826230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:54.826503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:54.827138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:54.828181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:54.828181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Sep 05 16:57:54.828181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: Sep 05 16:57:54.829097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:54.842908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 751/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 16:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:57:54.847244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:54.848813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:54.861561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76b5bf-5507-42c0-bbaa-eccf884ebd13 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:54.892829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:54.976747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8fc80-7147-4fbf-bddb-964998abeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:54.977348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8fc80-7147-4fbf-bddb-964998abeef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:54.977650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8fc80-7147-4fbf-bddb-964998abeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:54.977929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8fc80-7147-4fbf-bddb-964998abeef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:54.978903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8fc80-7147-4fbf-bddb-964998abeef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:54.979914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-1dd8fc80-7147-4fbf-bddb-964998abeef0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:54.979914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 16:57:54.979914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 16:57:54.980838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8fc80-7147-4fbf-bddb-964998abeef0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:54.994119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 749/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 16:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 16:57:54.998607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:55.000297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:55.036349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e0832-0bf6-4007-af14-143fd4405210 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:55.141662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb55774-a5e0-4fcd-8b16-ab539f30d2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:55.142221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55774-a5e0-4fcd-8b16-ab539f30d2bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:55.142496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55774-a5e0-4fcd-8b16-ab539f30d2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:55.142764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55774-a5e0-4fcd-8b16-ab539f30d2bd admin admin] PATH_INFO: `/v3/projects/a4275b62f02e47479a7aa3336fa56b4d/users/accfdd2b44594d24b4f062198532f1a6/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:55.168685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb55774-a5e0-4fcd-8b16-ab539f30d2bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4275b62f02e47479a7aa3336fa56b4d, user_id=accfdd2b44594d24b4f062198532f1a6, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:55.170161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-fcb55774-a5e0-4fcd-8b16-ab539f30d2bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:55.170161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Sep 05 16:57:55.170161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: Sep 05 16:57:55.173205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb55774-a5e0-4fcd-8b16-ab539f30d2bd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:55.206082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 752/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 16:57:54 2022] PUT /identity/v3/projects/a4275b62f02e47479a7aa3336fa56b4d/users/accfdd2b44594d24b4f062198532f1a6/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:57:55.219615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3969cfb-2485-414d-b023-33f456a5c069 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:55.219906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3969cfb-2485-414d-b023-33f456a5c069 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:55.220174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3969cfb-2485-414d-b023-33f456a5c069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:55.282083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e3969cfb-2485-414d-b023-33f456a5c069 None None] MFA Rules not processed for user `accfdd2b44594d24b4f062198532f1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:57:55.283289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e3969cfb-2485-414d-b023-33f456a5c069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:55.364705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 750/1502] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 16:57:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:57:55.378852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52376-deba-4991-b363-1944c4de051f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:55.379143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52376-deba-4991-b363-1944c4de051f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:55.379422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad52376-deba-4991-b363-1944c4de051f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:55.381245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 753/1503] 10.222.0.23 () {54 vars in 964 bytes} [Mon Sep 5 16:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:57:55.386452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf4f8-8931-499d-b78e-9fee21b74bfa None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:55.386735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf4f8-8931-499d-b78e-9fee21b74bfa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:55.387005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf4f8-8931-499d-b78e-9fee21b74bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:55.429947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:55.483369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-45adf4f8-8931-499d-b78e-9fee21b74bfa None None] MFA Rules not processed for user `36c795d7c6bc4c519c15f32489c44583`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:57:55.484634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-45adf4f8-8931-499d-b78e-9fee21b74bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:55.570904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 751/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Mon Sep 5 16:57:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:57:55.578984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52376-deba-4991-b363-1944c4de051f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:55.580568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ad52376-deba-4991-b363-1944c4de051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:55.637757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:55.697092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad52376-deba-4991-b363-1944c4de051f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:55.781550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:55.782171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:55.782432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:55.782698 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:55.784300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:55.895756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:55.897271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:55.900686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:55.923088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 754/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:57:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:57:55.968694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b41d0-d680-427b-9b46-789aee1e805c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:55.969121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b41d0-d680-427b-9b46-789aee1e805c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:55.969236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b41d0-d680-427b-9b46-789aee1e805c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:55.970549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 752/1506] 10.222.0.23 () {52 vars in 830 bytes} [Mon Sep 5 16:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:57:55.974254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:55.975739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:56.090217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfc4a0-52e9-492f-9cc4-6e791db483d8 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:56.192300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:56.192931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:56.193237 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:56.193514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] PATH_INFO: `/v3/users/accfdd2b44594d24b4f062198532f1a6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:56.207886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=accfdd2b44594d24b4f062198532f1a6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:56.209219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:56.209219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Sep 05 16:57:56.209219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: Sep 05 16:57:56.210460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:56.221301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:56.282632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'accfdd2b44594d24b4f062198532f1a6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:56.295896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'accfdd2b44594d24b4f062198532f1a6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:56.326875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'accfdd2b44594d24b4f062198532f1a6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:56.327426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'accfdd2b44594d24b4f062198532f1a6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:56.342175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'accfdd2b44594d24b4f062198532f1a6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:56.348412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 755/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 16:57:55 2022] DELETE /identity/v3/users/accfdd2b44594d24b4f062198532f1a6 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 16:57:56.353678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:56.354208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:56.354507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:56.416121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:57:56.417359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:56.505364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:56.521643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 753/1508] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 16:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:57:56.531924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:56.533289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:56.637878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b06eb-b792-45ee-8607-9279a2504bc3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:56.698480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:57:56.719244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d028c31-b087-401a-abc0-62752bf99808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:56.719894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028c31-b087-401a-abc0-62752bf99808 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:56.720174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028c31-b087-401a-abc0-62752bf99808 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:56.720479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028c31-b087-401a-abc0-62752bf99808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:56.722154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2d028c31-b087-401a-abc0-62752bf99808 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:56.846447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d028c31-b087-401a-abc0-62752bf99808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:56.847905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d028c31-b087-401a-abc0-62752bf99808 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:56.869244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 756/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 16:57:57.310107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:57:57.311338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:57:57.323427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5a1f3-c962-4d23-88f0-282fbcf0601c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:57:57.431854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:57:57.432552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:57:57.432821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:57:57.433105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] PATH_INFO: `/v3/projects/a4275b62f02e47479a7aa3336fa56b4d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:57:57.445503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4275b62f02e47479a7aa3336fa56b4d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:57:57.446558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 16:57:57.446558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 16:57:57.446558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 16:57:57.447773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:57:57.509983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4275b62f02e47479a7aa3336fa56b4d', 'request_id': 'req-90c62ec5-f925-4154-a612-f4a47dd7880d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:57.516771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4275b62f02e47479a7aa3336fa56b4d', 'request_id': 'req-90c62ec5-f925-4154-a612-f4a47dd7880d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:57.530998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4275b62f02e47479a7aa3336fa56b4d', 'request_id': 'req-90c62ec5-f925-4154-a612-f4a47dd7880d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:57.560172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] The token cache is being invalidate because project a4275b62f02e47479a7aa3336fa56b4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 16:57:57.560484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:57.560996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90c62ec5-f925-4154-a612-f4a47dd7880d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 16:57:57.562414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 754/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 16:57:57 2022] DELETE /identity/v3/projects/a4275b62f02e47479a7aa3336fa56b4d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 16:58:06.866621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=62 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:58:07.558101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:59:28.586876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc503e-563a-45cc-bfff-2ff892857ce8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:28.587496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc503e-563a-45cc-bfff-2ff892857ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:28.587496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc503e-563a-45cc-bfff-2ff892857ce8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:28.588852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:28.595139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37c064-bef7-449b-b239-fb3012aa57bd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:28.595611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37c064-bef7-449b-b239-fb3012aa57bd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:28.595720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37c064-bef7-449b-b239-fb3012aa57bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:28.691592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ed37c064-bef7-449b-b239-fb3012aa57bd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:28.692937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ed37c064-bef7-449b-b239-fb3012aa57bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:28.808565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:28.974557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc503e-563a-45cc-bfff-2ff892857ce8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:28.977503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a6dc503e-563a-45cc-bfff-2ff892857ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:29.113549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc503e-563a-45cc-bfff-2ff892857ce8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:29.119741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84879b-6704-4f1f-a397-21bea92be0bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:29.120383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84879b-6704-4f1f-a397-21bea92be0bc service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:29.120658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84879b-6704-4f1f-a397-21bea92be0bc service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:29.120932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84879b-6704-4f1f-a397-21bea92be0bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:29.129475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84879b-6704-4f1f-a397-21bea92be0bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:29.131060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84879b-6704-4f1f-a397-21bea92be0bc service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:29.140750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 758/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:29.939766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759336bd-a2c2-44ba-8342-d736f0485b92 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:29.940451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759336bd-a2c2-44ba-8342-d736f0485b92 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:29.940451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759336bd-a2c2-44ba-8342-d736f0485b92 None None] PATH_INFO: `/v3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:29.941863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 756/1514] 10.222.0.50 () {54 vars in 925 bytes} [Mon Sep 5 16:59:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 16:59:29.949613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6321863-842c-4414-a8de-511afe79751c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:29.950070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6321863-842c-4414-a8de-511afe79751c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:29.950242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6321863-842c-4414-a8de-511afe79751c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:29.959781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1d51-daaa-4e2e-b006-a74c4c0d9b80 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:29.960070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1d51-daaa-4e2e-b006-a74c4c0d9b80 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:29.960440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d1d51-daaa-4e2e-b006-a74c4c0d9b80 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:29.962071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 757/1515] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 16:59:29.966936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae354bd-0f8c-43d0-a1fa-f5d951c94ee3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:29.967223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae354bd-0f8c-43d0-a1fa-f5d951c94ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:29.967507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae354bd-0f8c-43d0-a1fa-f5d951c94ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:29.990350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2ae354bd-0f8c-43d0-a1fa-f5d951c94ee3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:29.991977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2ae354bd-0f8c-43d0-a1fa-f5d951c94ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:30.000756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 758/1516] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 16:59:30.019068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b6321863-842c-4414-a8de-511afe79751c None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:30.020315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b6321863-842c-4414-a8de-511afe79751c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:30.087771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:59:30.094014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 759/1517] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Sep 5 16:59:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:30.160537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae354bd-0f8c-43d0-a1fa-f5d951c94ee3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:30.181076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae354bd-0f8c-43d0-a1fa-f5d951c94ee3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:30.189024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15f4ae-6a60-4b25-8b24-4620b4f84c2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:30.189629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15f4ae-6a60-4b25-8b24-4620b4f84c2b service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:30.189975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15f4ae-6a60-4b25-8b24-4620b4f84c2b service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:30.190165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15f4ae-6a60-4b25-8b24-4620b4f84c2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:30.199912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15f4ae-6a60-4b25-8b24-4620b4f84c2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:30.201606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15f4ae-6a60-4b25-8b24-4620b4f84c2b service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:30.211722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 759/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:39.039922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9463-3200-47eb-a1d5-305f8e5841c5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:39.040567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9463-3200-47eb-a1d5-305f8e5841c5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:39.040567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9463-3200-47eb-a1d5-305f8e5841c5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:39.041842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 760/1519] 10.222.0.23 () {54 vars in 961 bytes} [Mon Sep 5 16:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:39.047309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7be46a-3e2f-4f34-aedd-9adf1ab37d0f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:39.047781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7be46a-3e2f-4f34-aedd-9adf1ab37d0f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:39.047883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7be46a-3e2f-4f34-aedd-9adf1ab37d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:39.068406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bd7be46a-3e2f-4f34-aedd-9adf1ab37d0f None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:39.069605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bd7be46a-3e2f-4f34-aedd-9adf1ab37d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:39.079269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 760/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Sep 5 16:59:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:39.087109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9463-3200-47eb-a1d5-305f8e5841c5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:39.102480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9463-3200-47eb-a1d5-305f8e5841c5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:39.108194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1a11d-6342-4e24-b489-d3d04bf7aeb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:39.108827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1a11d-6342-4e24-b489-d3d04bf7aeb7 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:39.109102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1a11d-6342-4e24-b489-d3d04bf7aeb7 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:39.109373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1a11d-6342-4e24-b489-d3d04bf7aeb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:39.117618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1a11d-6342-4e24-b489-d3d04bf7aeb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:39.119204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1a11d-6342-4e24-b489-d3d04bf7aeb7 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:39.128733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 761/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:41.434007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd457e51-d9b4-40ac-a3f7-1c68f551e7d3 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:41.434613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd457e51-d9b4-40ac-a3f7-1c68f551e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:41.434613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd457e51-d9b4-40ac-a3f7-1c68f551e7d3 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:41.436719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:59:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:41.441847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698067d0-0d5c-4824-9d0e-04874a632c36 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:41.442297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698067d0-0d5c-4824-9d0e-04874a632c36 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:41.442405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698067d0-0d5c-4824-9d0e-04874a632c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:41.463135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-698067d0-0d5c-4824-9d0e-04874a632c36 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:41.464394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-698067d0-0d5c-4824-9d0e-04874a632c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:41.473724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:41.627771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd457e51-d9b4-40ac-a3f7-1c68f551e7d3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:41.650582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd457e51-d9b4-40ac-a3f7-1c68f551e7d3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:41.658913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34742f9e-3a91-4d54-9b11-fdc4c296e120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:41.659514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34742f9e-3a91-4d54-9b11-fdc4c296e120 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:41.659788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34742f9e-3a91-4d54-9b11-fdc4c296e120 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:41.660060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34742f9e-3a91-4d54-9b11-fdc4c296e120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:41.668057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34742f9e-3a91-4d54-9b11-fdc4c296e120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:41.669973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34742f9e-3a91-4d54-9b11-fdc4c296e120 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:41.679456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 762/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:42.513128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd89925-c3d8-4cfb-82b7-190e30b36726 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:42.513754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd89925-c3d8-4cfb-82b7-190e30b36726 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:42.513754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd89925-c3d8-4cfb-82b7-190e30b36726 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:42.515881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 5 16:59:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:42.521129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219082ad-25d1-4be1-b869-b5c623bb74b0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:42.521620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219082ad-25d1-4be1-b869-b5c623bb74b0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:42.521620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219082ad-25d1-4be1-b869-b5c623bb74b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:42.546336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-219082ad-25d1-4be1-b869-b5c623bb74b0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:42.547586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-219082ad-25d1-4be1-b869-b5c623bb74b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:42.555615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 5 16:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:42.717216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd89925-c3d8-4cfb-82b7-190e30b36726 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:42.738741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd89925-c3d8-4cfb-82b7-190e30b36726 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:42.744647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8256b1c-f25b-471d-984d-59e62c8ef7ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:42.745295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8256b1c-f25b-471d-984d-59e62c8ef7ed service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:42.745573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8256b1c-f25b-471d-984d-59e62c8ef7ed service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:42.745843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8256b1c-f25b-471d-984d-59e62c8ef7ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:42.758377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8256b1c-f25b-471d-984d-59e62c8ef7ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:42.763868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8256b1c-f25b-471d-984d-59e62c8ef7ed service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:42.777384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 764/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:46.784687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e42ea-797d-4e61-ad05-db87c8c7e7d1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:46.785335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e42ea-797d-4e61-ad05-db87c8c7e7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:46.785335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e42ea-797d-4e61-ad05-db87c8c7e7d1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:46.786614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:46.793083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5e76-12da-40a8-a634-a31cef7bc4b5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:46.794421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5e76-12da-40a8-a634-a31cef7bc4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:46.794883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e5e76-12da-40a8-a634-a31cef7bc4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:46.822307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a85e5e76-12da-40a8-a634-a31cef7bc4b5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:46.823503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a85e5e76-12da-40a8-a634-a31cef7bc4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:46.831606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:47.114798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e42ea-797d-4e61-ad05-db87c8c7e7d1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:47.116773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c81e42ea-797d-4e61-ad05-db87c8c7e7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:47.149688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 16:59:47.204924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e42ea-797d-4e61-ad05-db87c8c7e7d1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:47.209356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f7760-d3ff-426d-aa58-d8613fd1016c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:47.209944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f7760-d3ff-426d-aa58-d8613fd1016c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:47.210222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f7760-d3ff-426d-aa58-d8613fd1016c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:47.210494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f7760-d3ff-426d-aa58-d8613fd1016c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:47.219793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f7760-d3ff-426d-aa58-d8613fd1016c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:47.221275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f7760-d3ff-426d-aa58-d8613fd1016c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:47.231186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 765/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:49.145298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a7bc-c5ca-4337-98ed-1afff4eee8e9 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:49.146177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a7bc-c5ca-4337-98ed-1afff4eee8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:49.146177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543a7bc-c5ca-4337-98ed-1afff4eee8e9 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:49.148455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:59:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:49.156791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb003a3-1917-431b-8318-7bf96322ac0e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:49.157229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb003a3-1917-431b-8318-7bf96322ac0e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:49.157433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb003a3-1917-431b-8318-7bf96322ac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:49.187632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cdb003a3-1917-431b-8318-7bf96322ac0e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:49.188899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cdb003a3-1917-431b-8318-7bf96322ac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:49.197943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:49.495684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a7bc-c5ca-4337-98ed-1afff4eee8e9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:49.508833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543a7bc-c5ca-4337-98ed-1afff4eee8e9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:49.513697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240db18-f78e-4266-902f-167a6cdb2b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:49.514484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240db18-f78e-4266-902f-167a6cdb2b0c service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:49.514833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240db18-f78e-4266-902f-167a6cdb2b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:49.515186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240db18-f78e-4266-902f-167a6cdb2b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:49.525179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240db18-f78e-4266-902f-167a6cdb2b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:49.526623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240db18-f78e-4266-902f-167a6cdb2b0c service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:49.536936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 767/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:53.340180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce213049-dc1b-401f-96d5-2ac1eee09305 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:53.340780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce213049-dc1b-401f-96d5-2ac1eee09305 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:53.340780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce213049-dc1b-401f-96d5-2ac1eee09305 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:53.342123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:53.348446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4167d-d725-4971-83b6-baf5a591f6cb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:53.348967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4167d-d725-4971-83b6-baf5a591f6cb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:53.348967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4167d-d725-4971-83b6-baf5a591f6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:53.374128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ddc4167d-d725-4971-83b6-baf5a591f6cb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:53.375504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ddc4167d-d725-4971-83b6-baf5a591f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:53.384176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:53.937855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce213049-dc1b-401f-96d5-2ac1eee09305 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:53.941279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ce213049-dc1b-401f-96d5-2ac1eee09305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:54.030674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce213049-dc1b-401f-96d5-2ac1eee09305 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:54.034764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1d812-b746-445a-8e39-7ef3c1c3d6eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:54.035376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1d812-b746-445a-8e39-7ef3c1c3d6eb service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:54.035657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1d812-b746-445a-8e39-7ef3c1c3d6eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:54.035931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1d812-b746-445a-8e39-7ef3c1c3d6eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:54.047290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1d812-b746-445a-8e39-7ef3c1c3d6eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:54.048801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1d812-b746-445a-8e39-7ef3c1c3d6eb service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:54.058747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 768/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 16:59:55.968855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e363-ca7e-4442-9978-3d1ef0be42b5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:55.970123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e363-ca7e-4442-9978-3d1ef0be42b5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:55.970123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e363-ca7e-4442-9978-3d1ef0be42b5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:55.972072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 5 16:59:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 16:59:55.978249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f95a-39b1-42dc-aa88-b0d67a02ba5a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:55.978711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f95a-39b1-42dc-aa88-b0d67a02ba5a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:55.978813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245f95a-39b1-42dc-aa88-b0d67a02ba5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:56.000642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b245f95a-39b1-42dc-aa88-b0d67a02ba5a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 16:59:56.001857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b245f95a-39b1-42dc-aa88-b0d67a02ba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:56.010625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 5 16:59:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 16:59:56.561795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e363-ca7e-4442-9978-3d1ef0be42b5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 16:59:56.564208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1389e363-ca7e-4442-9978-3d1ef0be42b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 16:59:56.579577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e363-ca7e-4442-9978-3d1ef0be42b5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 16:59:56.583993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0562f-dce7-491a-b4fa-b5345b0e2657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 16:59:56.584617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0562f-dce7-491a-b4fa-b5345b0e2657 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 16:59:56.584896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0562f-dce7-491a-b4fa-b5345b0e2657 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 16:59:56.585160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0562f-dce7-491a-b4fa-b5345b0e2657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 16:59:56.595180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0562f-dce7-491a-b4fa-b5345b0e2657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 16:59:56.596663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0562f-dce7-491a-b4fa-b5345b0e2657 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 16:59:56.606540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 770/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 16:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:00:05.995289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=66 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:00:06.602253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=34 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:22.994654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391af24-010d-439a-a0d4-b3f213e5bbd7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:22.995839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391af24-010d-439a-a0d4-b3f213e5bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:22.996362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391af24-010d-439a-a0d4-b3f213e5bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.023794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0391af24-010d-439a-a0d4-b3f213e5bbd7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.025445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0391af24-010d-439a-a0d4-b3f213e5bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.038175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 770/1540] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.047800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074fb5c-bd85-48db-9932-9a69d6c3d205 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.048975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074fb5c-bd85-48db-9932-9a69d6c3d205 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.049466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074fb5c-bd85-48db-9932-9a69d6c3d205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.077351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9074fb5c-bd85-48db-9932-9a69d6c3d205 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.078961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9074fb5c-bd85-48db-9932-9a69d6c3d205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.109724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 771/1541] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.117406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391af24-010d-439a-a0d4-b3f213e5bbd7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.131056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391af24-010d-439a-a0d4-b3f213e5bbd7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.133980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e1ac-3f1e-4f04-b89d-d1464f8c8c1a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.134620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e1ac-3f1e-4f04-b89d-d1464f8c8c1a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.135134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1e1ac-3f1e-4f04-b89d-d1464f8c8c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.144406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5420f-f015-4573-b2d7-409a31cf40e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.147513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5420f-f015-4573-b2d7-409a31cf40e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.148062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5420f-f015-4573-b2d7-409a31cf40e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.150047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5420f-f015-4573-b2d7-409a31cf40e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.151100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5420f-f015-4573-b2d7-409a31cf40e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.151627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5420f-f015-4573-b2d7-409a31cf40e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.153726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5420f-f015-4573-b2d7-409a31cf40e6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.180616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 771/1542] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.181022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-aea1e1ac-3f1e-4f04-b89d-d1464f8c8c1a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.182695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-aea1e1ac-3f1e-4f04-b89d-d1464f8c8c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.184222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a9ac-a127-4e01-b645-3d3af8622611 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.185599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a9ac-a127-4e01-b645-3d3af8622611 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.187259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91a9ac-a127-4e01-b645-3d3af8622611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.199623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 772/1543] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.201350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c7de-1e3f-42a8-b9df-8c35e60429b0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.201862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c7de-1e3f-42a8-b9df-8c35e60429b0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.202336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1c7de-1e3f-42a8-b9df-8c35e60429b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.224972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1a91a9ac-a127-4e01-b645-3d3af8622611 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.226177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1a91a9ac-a127-4e01-b645-3d3af8622611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.227739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0ef1c7de-1e3f-42a8-b9df-8c35e60429b0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.229316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0ef1c7de-1e3f-42a8-b9df-8c35e60429b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.236883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 772/1544] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.237673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a9ac-a127-4e01-b645-3d3af8622611 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.239591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 773/1545] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.241311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086cb9d-c377-49d6-9eac-bed95227d37f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.241885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086cb9d-c377-49d6-9eac-bed95227d37f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.242375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086cb9d-c377-49d6-9eac-bed95227d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.248397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91a9ac-a127-4e01-b645-3d3af8622611 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.253759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f48e5-b361-46b6-b601-f165dbd13eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.253939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f48e5-b361-46b6-b601-f165dbd13eec admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.254348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f48e5-b361-46b6-b601-f165dbd13eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.254475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f48e5-b361-46b6-b601-f165dbd13eec admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.255029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f48e5-b361-46b6-b601-f165dbd13eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.256046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-ec3f48e5-b361-46b6-b601-f165dbd13eec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 17:01:23.256046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 17:01:23.256046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 17:01:23.257349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f48e5-b361-46b6-b601-f165dbd13eec admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.265631 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8086cb9d-c377-49d6-9eac-bed95227d37f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.266847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8086cb9d-c377-49d6-9eac-bed95227d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.276707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 774/1546] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.278171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e84f-6632-4bd1-b03e-34e87ff08477 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.278458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e84f-6632-4bd1-b03e-34e87ff08477 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.278739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e84f-6632-4bd1-b03e-34e87ff08477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.299712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 773/1547] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:23.300823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f1d1e84f-6632-4bd1-b03e-34e87ff08477 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.301589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55691c-0580-4625-bcab-41d0db4497c0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.301881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55691c-0580-4625-bcab-41d0db4497c0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.302074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f1d1e84f-6632-4bd1-b03e-34e87ff08477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.302191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55691c-0580-4625-bcab-41d0db4497c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.323631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4a55691c-0580-4625-bcab-41d0db4497c0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.324891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4a55691c-0580-4625-bcab-41d0db4497c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.374134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 775/1548] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.375624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7f21d-c144-4b49-b78c-fe4078244f09 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.375914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7f21d-c144-4b49-b78c-fe4078244f09 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.376541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7f21d-c144-4b49-b78c-fe4078244f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.388387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 774/1549] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.389805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffd5e-be58-499a-8ce3-fba72308ea7d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.390247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffd5e-be58-499a-8ce3-fba72308ea7d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.390527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbffd5e-be58-499a-8ce3-fba72308ea7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.397259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-00e7f21d-c144-4b49-b78c-fe4078244f09 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.398477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-00e7f21d-c144-4b49-b78c-fe4078244f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.407029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 776/1550] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.408360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ffed0-eef8-4b77-842d-b5d68e67a681 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.408663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ffed0-eef8-4b77-842d-b5d68e67a681 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.408790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ffed0-eef8-4b77-842d-b5d68e67a681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.412611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dbbffd5e-be58-499a-8ce3-fba72308ea7d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.414041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dbbffd5e-be58-499a-8ce3-fba72308ea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.422238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 775/1551] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.423477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f66ca7-9e37-4e16-a5d4-c2b797b57ec1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.423755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f66ca7-9e37-4e16-a5d4-c2b797b57ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.424023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f66ca7-9e37-4e16-a5d4-c2b797b57ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.428732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-392ffed0-eef8-4b77-842d-b5d68e67a681 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.429921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-392ffed0-eef8-4b77-842d-b5d68e67a681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.438799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 777/1552] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.439699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ffed0-eef8-4b77-842d-b5d68e67a681 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.444161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-73f66ca7-9e37-4e16-a5d4-c2b797b57ec1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.445367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-73f66ca7-9e37-4e16-a5d4-c2b797b57ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.451446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ffed0-eef8-4b77-842d-b5d68e67a681 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.453936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 776/1553] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.455191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98dd35-177d-4c23-875e-8aff5a1e6a41 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.455499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98dd35-177d-4c23-875e-8aff5a1e6a41 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.455769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98dd35-177d-4c23-875e-8aff5a1e6a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.459792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea3079-1e7b-4ea6-9c32-df985ae2953b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.460430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3079-1e7b-4ea6-9c32-df985ae2953b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.460730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3079-1e7b-4ea6-9c32-df985ae2953b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.461000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3079-1e7b-4ea6-9c32-df985ae2953b admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.461652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea3079-1e7b-4ea6-9c32-df985ae2953b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.463486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-2dea3079-1e7b-4ea6-9c32-df985ae2953b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 17:01:23.463486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Sep 05 17:01:23.463486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: Sep 05 17:01:23.464768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea3079-1e7b-4ea6-9c32-df985ae2953b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.475332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6f98dd35-177d-4c23-875e-8aff5a1e6a41 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.476633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6f98dd35-177d-4c23-875e-8aff5a1e6a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.485843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 777/1554] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.486485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98dd35-177d-4c23-875e-8aff5a1e6a41 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.492245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 778/1555] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/users => generated 284 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:23.493817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae034d-f8d3-401c-ac6d-9a4105778215 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.494171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae034d-f8d3-401c-ac6d-9a4105778215 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.494452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae034d-f8d3-401c-ac6d-9a4105778215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.499782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98dd35-177d-4c23-875e-8aff5a1e6a41 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.504976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cb547-393f-4b2c-9f67-f085ca6be9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.505634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb547-393f-4b2c-9f67-f085ca6be9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.505960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb547-393f-4b2c-9f67-f085ca6be9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.506235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cb547-393f-4b2c-9f67-f085ca6be9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.506895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cb547-393f-4b2c-9f67-f085ca6be9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.507195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cb547-393f-4b2c-9f67-f085ca6be9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.508921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cb547-393f-4b2c-9f67-f085ca6be9a0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.514902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a4ae034d-f8d3-401c-ac6d-9a4105778215 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.516111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a4ae034d-f8d3-401c-ac6d-9a4105778215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.521706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 778/1556] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.523056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010ef94-28b7-45ed-87de-9f29d51a7169 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.523336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010ef94-28b7-45ed-87de-9f29d51a7169 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.523605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010ef94-28b7-45ed-87de-9f29d51a7169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.524437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 779/1557] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.525085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae034d-f8d3-401c-ac6d-9a4105778215 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.534639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae034d-f8d3-401c-ac6d-9a4105778215 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.539215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.539908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.540229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.540578 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.541232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.541563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.542590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 05 17:01:23.542590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Sep 05 17:01:23.542590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: Sep 05 17:01:23.543487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.549723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e010ef94-28b7-45ed-87de-9f29d51a7169 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.550949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e010ef94-28b7-45ed-87de-9f29d51a7169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.555422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 780/1558] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.556295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.566289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c1bb5-5972-4df4-bc3f-3d3c3579f910 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.567055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 779/1559] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.567946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010ef94-28b7-45ed-87de-9f29d51a7169 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.573916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1e658-b8d0-47d9-b761-dc14702d152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.574519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1e658-b8d0-47d9-b761-dc14702d152d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.574801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1e658-b8d0-47d9-b761-dc14702d152d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.575091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1e658-b8d0-47d9-b761-dc14702d152d admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.575724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1e658-b8d0-47d9-b761-dc14702d152d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.576035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1e658-b8d0-47d9-b761-dc14702d152d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.577534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010ef94-28b7-45ed-87de-9f29d51a7169 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.577687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1e658-b8d0-47d9-b761-dc14702d152d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.582044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.582627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.582903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.583176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.583803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.584101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.585754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.589854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 781/1560] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.591019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481983e-5f73-45f7-a3ac-c4b5ac5fd57d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.591331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481983e-5f73-45f7-a3ac-c4b5ac5fd57d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.591607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481983e-5f73-45f7-a3ac-c4b5ac5fd57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.602915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 780/1561] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.603800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.613529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d701dc3-58d1-4204-8eab-1e1f5e245623 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.615284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1481983e-5f73-45f7-a3ac-c4b5ac5fd57d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:23.616536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1481983e-5f73-45f7-a3ac-c4b5ac5fd57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:23.618268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.618906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.619226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.619578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.620251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.621860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.625772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 782/1562] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:23.626549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481983e-5f73-45f7-a3ac-c4b5ac5fd57d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.632622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 781/1563] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:23.633397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.641165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481983e-5f73-45f7-a3ac-c4b5ac5fd57d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.643548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd668d13-fc5f-4c75-9213-2e4e62b833c2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.646579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.647188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.647468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.647739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.648346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.648492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.648713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.648931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.649206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.649478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.650289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.650480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.652343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.663101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 783/1564] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.663960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.676459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b9cd-1052-4507-a334-29c408954f49 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.681470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.682086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.682366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.682636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.683273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.684165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 782/1565] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:23.685152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.685579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.706337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:23.717427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 784/1566] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:23.718251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.749103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:23.755040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295c164e-a28c-4b3d-988a-b1b86746e5f3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.782415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aee80d-4b9c-42ba-a6dd-0c80208114b3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.787041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.787663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.787941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.788252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.788883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.790944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.811278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.812004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.812362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.812659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.813307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.813624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.815252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.818592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 785/1567] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:23.819477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.834350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 783/1568] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.834350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.838239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cfb3e-32d6-4277-8b16-5e3681cdcce3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.890332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc59f29-aef6-47e8-9a4e-a07c2a1e65ef admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.895047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.895653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.895932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.896255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.896867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.898065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.898681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.898959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.899094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.899228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.899921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:23.900275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.901948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.913511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 784/1569] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:23.914501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.924504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d82c6-eb4d-4abd-bbf2-508609a2ab83 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.929039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 786/1570] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:23.929287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:23.929843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.929968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:23.930162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:23.930420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:23.931485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:23.933291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:23.945170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 785/1571] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:23.945835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:23.955312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6b2d1-c2de-4a27-a6c6-0a50825682e6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.994560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39941907-17db-45c5-9ae3-d4a42cbafe62 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:23.999276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:24.000780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.001546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.001852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.002213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.002960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.005001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.016521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.017074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.017650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.017650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.018234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.020194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.034687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 787/1572] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.035502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.046538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 786/1573] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:23 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.049063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.059232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889458b-a803-4286-b21a-339fc506a338 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.062177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:24.099660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac313252-967b-486f-93e6-71c81ad4686f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.105031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.105674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.105955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.106207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.106832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.108801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.118778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.119492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.119778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.120051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.120789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.122919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.131618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 788/1574] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.132612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.144274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af5208-0b5e-4168-b63a-b5f1af5d4740 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.146238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 787/1575] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.146883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.149006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.149685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.150100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.150626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.151422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.153441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.158157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881c7d2-6de9-458f-9fba-8f88c8c7d9c6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.163723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.163968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.164289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.164619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.165268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.167180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.182741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 789/1576] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.183630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.200363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 788/1577] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/projects => generated 357 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.201241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.245640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f738b-b47c-4e1b-b634-41b55660fecd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.263657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a0536-be94-4ee4-9c09-7f0d07211a4b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.269146 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.269751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.270028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.270299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] PATH_INFO: `/v3/projects/68ace03f2433467299f56d5392dff17e/users/c7c5ae21a0ef42a49c14d2665a2d5752/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.285336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ace03f2433467299f56d5392dff17e, user_id=c7c5ae21a0ef42a49c14d2665a2d5752, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.286447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 17:01:24.286447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 17:01:24.286447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 17:01:24.288792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.300162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.300871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.301156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.301460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.302100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.308566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 789/1578] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:24 2022] PUT /identity/v3/projects/68ace03f2433467299f56d5392dff17e/users/c7c5ae21a0ef42a49c14d2665a2d5752/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:24.308728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.308812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.329659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 790/1579] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.330769 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.340312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:24.342472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98956-1f01-44d4-8773-7c21068de63e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.377728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:24.378784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e2214-db9b-4c46-91d4-f68fadfa9f1e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.383657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.384352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.384634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.384907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.385688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.388301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.403607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.404284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.404563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.404886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.405536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.407157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.412577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 790/1580] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.413560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.421031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 791/1581] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.421869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.426558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152914a5-2f68-4b3f-85e3-e503d3bb6606 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.431556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.432170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.432455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.432579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0666-5a5d-4a36-884d-c333911114fe admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.432731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.433358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.434962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.438091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.438687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.438964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.439232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.439925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.441623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.445955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 791/1582] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.447273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.456461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 792/1583] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.456834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcd84f-dbf3-48db-b609-2dff4a4d18b4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.457272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.463702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358a3c8-172f-4fd4-b309-35be0b5edf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.464315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a3c8-172f-4fd4-b309-35be0b5edf5d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.464607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a3c8-172f-4fd4-b309-35be0b5edf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.464890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358a3c8-172f-4fd4-b309-35be0b5edf5d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.465516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358a3c8-172f-4fd4-b309-35be0b5edf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.467467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358a3c8-172f-4fd4-b309-35be0b5edf5d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.468920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db831c-95e1-435e-bf16-5de3998578cb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.473598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.474202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.474479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.474748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.475376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.477358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.493457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 792/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/users => generated 282 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.495273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37c8b-0c22-4fa9-8975-fda38893d1a5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.495560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37c8b-0c22-4fa9-8975-fda38893d1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.495831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e37c8b-0c22-4fa9-8975-fda38893d1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.503495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 793/1585] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/users => generated 279 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:24.504421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.514569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b45d19-b595-4366-af27-3549c50f36cb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.519112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.519884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.520249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.520523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.521179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.522782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.527605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f6e37c8b-0c22-4fa9-8975-fda38893d1a5 None None] MFA Rules not processed for user `c7c5ae21a0ef42a49c14d2665a2d5752`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:24.528852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f6e37c8b-0c22-4fa9-8975-fda38893d1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:24.533333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 794/1586] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.534276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.543990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47eb060-8f9c-4c6a-b0fc-e84f9680ae54 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.548551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.549140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.549415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.549690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.550308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.551890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.562014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 795/1587] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.565044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.575596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13c00e-20c8-4052-9e81-1dc294a5e2e4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.580570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.581317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.581593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.581876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.582480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.584120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.594123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 796/1588] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.594927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.595270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 793/1589] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:24.596003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e37c8b-0c22-4fa9-8975-fda38893d1a5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.610267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c100d7-e0d9-4ea7-9ab9-8cb5a01d7613 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.610794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e37c8b-0c22-4fa9-8975-fda38893d1a5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.614808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.615413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.615689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.615959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.616131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.616620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.616783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.617061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.617332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.617983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.618226 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.619745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.629114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 797/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.629908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.630653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 794/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.631455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.642532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699e4d2-91c9-45de-b6a2-64d32db5a409 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.642900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f323d-62c2-466f-90a4-27aabb541b63 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.647207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.647806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.648085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.648342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.648473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.648932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.649031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.649210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.649479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.650185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.651291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.651857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.662035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 798/1592] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.665311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 795/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.665492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.665602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.673383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b870-4712-43ed-bf64-79e3d828d3ab admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.673683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385790c-be60-4001-93af-df24903f5ee0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.677861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.678230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.678559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.678814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.678921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.679096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.679199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.679362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.679718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.679986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.681567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.681694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.691906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 799/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.692519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 796/1595] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.692933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.693291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.703333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef7a8b-8bdd-462d-92e8-a63410e01f33 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.707909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.708594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.708876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.709152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] PATH_INFO: `/v3/projects/76544b1e49f748bcbd66fbb7d459f7d5/users/eb81051fdb914d6abd9f5f2db542e564/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.725412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] RBAC: Authorizing `identity:create_grant(project_id=76544b1e49f748bcbd66fbb7d459f7d5, user_id=eb81051fdb914d6abd9f5f2db542e564, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.728506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.739518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:24.743210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 800/1596] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:24 2022] PUT /identity/v3/projects/76544b1e49f748bcbd66fbb7d459f7d5/users/eb81051fdb914d6abd9f5f2db542e564/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:24.744475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.753070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d5105-7060-43bd-922a-0f9287eebd3a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.803379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.803908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.804207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.804482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.815690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736622fb-be31-48e4-84e4-f1e40d87660e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.820427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.821092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.821369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.821665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] PATH_INFO: `/v3/projects/e933bc308b31482a9db533a7e89aeb4b/users/c53ac49fbc054c01a5a1ef41e04bf253/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.833029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:24.836779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e933bc308b31482a9db533a7e89aeb4b, user_id=c53ac49fbc054c01a5a1ef41e04bf253, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.840855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.856333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 801/1597] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:24 2022] PUT /identity/v3/projects/e933bc308b31482a9db533a7e89aeb4b/users/c53ac49fbc054c01a5a1ef41e04bf253/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:24.857507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.863603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.865081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.874262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 797/1598] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:24.874992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.884969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1d6e0-1880-4b81-85da-6c4afbd70a4d service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.924792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffda6e8-0f1d-4bd5-b7c5-f25c8ce7b4c8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.929744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.930561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.930843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.931302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] PATH_INFO: `/v3/projects/bdc64dc236bf4a87b93a1561ff53293b/users/e2d470c0a2624ddcbb015ca02598e5d4/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.944141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:24.944970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:24.945317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:24.945884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:24.946362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.948233 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.949148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc64dc236bf4a87b93a1561ff53293b, user_id=e2d470c0a2624ddcbb015ca02598e5d4, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:24.952052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:24.959035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 798/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:24.959956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:24.970173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10163fb7-c0c5-4ca4-9aba-97d29e542043 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:24.973985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 802/1600] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:24 2022] PUT /identity/v3/projects/bdc64dc236bf4a87b93a1561ff53293b/users/e2d470c0a2624ddcbb015ca02598e5d4/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:24.974839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.031839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:25.046246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881ad60-c701-423a-a320-6063a9e9d7fd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.047023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.047783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.048065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.048382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.049066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.050700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.052530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b8ce2-84ff-436a-b579-5125ddb48c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.054362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b8ce2-84ff-436a-b579-5125ddb48c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.054362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b8ce2-84ff-436a-b579-5125ddb48c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.054855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b8ce2-84ff-436a-b579-5125ddb48c12 admin admin] PATH_INFO: `/v3/projects/cdbd9f06f3484da6a6fd14977b82f384/users/06e60909a8b04008af659620527e3236/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.065095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 799/1601] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:25.065095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.072501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b8ce2-84ff-436a-b579-5125ddb48c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdbd9f06f3484da6a6fd14977b82f384, user_id=06e60909a8b04008af659620527e3236, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.075636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b8ce2-84ff-436a-b579-5125ddb48c12 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.078422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c06533-458e-4646-835f-7605ee309daa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.083753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe03ae1-409c-45a4-902c-0561c7b1b24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.084516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03ae1-409c-45a4-902c-0561c7b1b24a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.084852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03ae1-409c-45a4-902c-0561c7b1b24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.085226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03ae1-409c-45a4-902c-0561c7b1b24a admin admin] PATH_INFO: `/v3/projects/3fba05bd3cd4401597809b1fd0cdb439/users/117a819351b742deb901c081d0a4347b/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.093961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 803/1602] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:24 2022] PUT /identity/v3/projects/cdbd9f06f3484da6a6fd14977b82f384/users/06e60909a8b04008af659620527e3236/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:25.095257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6102fb-fc37-44e1-90d8-fdd4b25c6631 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.095633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6102fb-fc37-44e1-90d8-fdd4b25c6631 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.095994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6102fb-fc37-44e1-90d8-fdd4b25c6631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.101955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe03ae1-409c-45a4-902c-0561c7b1b24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fba05bd3cd4401597809b1fd0cdb439, user_id=117a819351b742deb901c081d0a4347b, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.105858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe03ae1-409c-45a4-902c-0561c7b1b24a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.123287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 800/1603] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:25 2022] PUT /identity/v3/projects/3fba05bd3cd4401597809b1fd0cdb439/users/117a819351b742deb901c081d0a4347b/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:25.124347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf0be5-d581-4899-932e-e014e464c6dd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.124627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf0be5-d581-4899-932e-e014e464c6dd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.125059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf0be5-d581-4899-932e-e014e464c6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.128973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2d6102fb-fc37-44e1-90d8-fdd4b25c6631 None None] MFA Rules not processed for user `eb81051fdb914d6abd9f5f2db542e564`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.130293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2d6102fb-fc37-44e1-90d8-fdd4b25c6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.145422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9faf0be5-d581-4899-932e-e014e464c6dd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.146739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9faf0be5-d581-4899-932e-e014e464c6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.189133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:25.213624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 804/1604] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.216614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 801/1605] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.221884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33281fd-15cc-42d8-8a61-52abf14ce4f9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.221884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33281fd-15cc-42d8-8a61-52abf14ce4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.221884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33281fd-15cc-42d8-8a61-52abf14ce4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.222182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6102fb-fc37-44e1-90d8-fdd4b25c6631 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.233287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6102fb-fc37-44e1-90d8-fdd4b25c6631 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.238660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.239466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.239848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.240343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] PATH_INFO: `/v3/projects/adfd09010e3446bf998019b95c890156/users/212e5af7a7674a16b1ca74bab4da5309/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.256137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adfd09010e3446bf998019b95c890156, user_id=212e5af7a7674a16b1ca74bab4da5309, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.257979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b33281fd-15cc-42d8-8a61-52abf14ce4f9 None None] MFA Rules not processed for user `e2d470c0a2624ddcbb015ca02598e5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.259178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.259334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b33281fd-15cc-42d8-8a61-52abf14ce4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.278302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 805/1606] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:25 2022] PUT /identity/v3/projects/adfd09010e3446bf998019b95c890156/users/212e5af7a7674a16b1ca74bab4da5309/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:25.279320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.336117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 802/1607] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.337817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf1b5-68d1-41d3-b396-5755a9ff34b8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.338782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf1b5-68d1-41d3-b396-5755a9ff34b8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.338782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf1b5-68d1-41d3-b396-5755a9ff34b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.357602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93cdc7-d267-4fe6-a267-a0a67d49ecb4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.360857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-77ccf1b5-68d1-41d3-b396-5755a9ff34b8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.362075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-77ccf1b5-68d1-41d3-b396-5755a9ff34b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.364137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.366847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.369199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.369672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] PATH_INFO: `/v3/projects/069563a168314ecea0dc0d87766a62ce/users/a78f0df187e5487385b5cf6579704d24/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.374183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 803/1608] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.375556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2767-3f25-4a39-961d-bd8d7d17ebe5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.375940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2767-3f25-4a39-961d-bd8d7d17ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.376445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e2767-3f25-4a39-961d-bd8d7d17ebe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.403342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6a1e2767-3f25-4a39-961d-bd8d7d17ebe5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.404649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6a1e2767-3f25-4a39-961d-bd8d7d17ebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.417817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069563a168314ecea0dc0d87766a62ce, user_id=a78f0df187e5487385b5cf6579704d24, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.421353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.424577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 804/1609] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.426725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2767-3f25-4a39-961d-bd8d7d17ebe5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.452211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e2767-3f25-4a39-961d-bd8d7d17ebe5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.453016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 806/1610] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:25 2022] PUT /identity/v3/projects/069563a168314ecea0dc0d87766a62ce/users/a78f0df187e5487385b5cf6579704d24/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:25.455024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.496085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:25.532453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966b310-3fdb-4ea4-8621-d98d44c61de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.532453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966b310-3fdb-4ea4-8621-d98d44c61de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.532453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966b310-3fdb-4ea4-8621-d98d44c61de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.532453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966b310-3fdb-4ea4-8621-d98d44c61de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.532955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966b310-3fdb-4ea4-8621-d98d44c61de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.534072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966b310-3fdb-4ea4-8621-d98d44c61de6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.543059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f67f2-8817-4f4d-88fe-b4ffeb1b2c02 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.546999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c5655-1d67-4b8d-acc3-ce5f623649f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.547740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c5655-1d67-4b8d-acc3-ce5f623649f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.548094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c5655-1d67-4b8d-acc3-ce5f623649f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.548457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c5655-1d67-4b8d-acc3-ce5f623649f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.558514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c5655-1d67-4b8d-acc3-ce5f623649f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.559982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c5655-1d67-4b8d-acc3-ce5f623649f6 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.565423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 805/1611] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:25.566848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96afc43-5cfa-45db-ba5b-d3d673f857fd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.567317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96afc43-5cfa-45db-ba5b-d3d673f857fd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.567740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96afc43-5cfa-45db-ba5b-d3d673f857fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.571957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 807/1612] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:25.571957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373e3d-d965-4fab-802a-2418227fa889 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.572424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373e3d-d965-4fab-802a-2418227fa889 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.572424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373e3d-d965-4fab-802a-2418227fa889 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.573553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 808/1613] 10.222.0.23 () {54 vars in 962 bytes} [Mon Sep 5 17:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:25.574375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15373e3d-d965-4fab-802a-2418227fa889 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.613325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e96afc43-5cfa-45db-ba5b-d3d673f857fd None None] MFA Rules not processed for user `212e5af7a7674a16b1ca74bab4da5309`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.614999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e96afc43-5cfa-45db-ba5b-d3d673f857fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.628197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:25.635576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15373e3d-d965-4fab-802a-2418227fa889 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.640857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feddaf72-e7d3-4be8-946a-2ee0fa1b2b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.641514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddaf72-e7d3-4be8-946a-2ee0fa1b2b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.641793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddaf72-e7d3-4be8-946a-2ee0fa1b2b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.642212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feddaf72-e7d3-4be8-946a-2ee0fa1b2b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.699145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 806/1614] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.700275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96afc43-5cfa-45db-ba5b-d3d673f857fd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.710554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96afc43-5cfa-45db-ba5b-d3d673f857fd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.715606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.716451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.716915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.717254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.722128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddaf72-e7d3-4be8-946a-2ee0fa1b2b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.724133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feddaf72-e7d3-4be8-946a-2ee0fa1b2b56 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.729380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.730838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.737844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 809/1615] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:25.739441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d4b2-06f6-4240-bb61-2615adfc5fca None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.740016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d4b2-06f6-4240-bb61-2615adfc5fca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.740470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d4b2-06f6-4240-bb61-2615adfc5fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.742993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 807/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:25.743736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.754345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a10f74-bea8-46cc-8bab-1ae89178b341 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.759375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e9368-5948-452a-9a70-9df2582454ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.760812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e9368-5948-452a-9a70-9df2582454ed admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.761132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e9368-5948-452a-9a70-9df2582454ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.761479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e9368-5948-452a-9a70-9df2582454ed admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.762145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e9368-5948-452a-9a70-9df2582454ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.764122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e9368-5948-452a-9a70-9df2582454ed admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.781477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fae8d4b2-06f6-4240-bb61-2615adfc5fca None None] MFA Rules not processed for user `a78f0df187e5487385b5cf6579704d24`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.782749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fae8d4b2-06f6-4240-bb61-2615adfc5fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.793915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 808/1617] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/users => generated 293 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:25.795371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11653-097b-4eed-ba1a-3c3aa8a493ed None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.795657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11653-097b-4eed-ba1a-3c3aa8a493ed None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.795928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb11653-097b-4eed-ba1a-3c3aa8a493ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.846074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6eb11653-097b-4eed-ba1a-3c3aa8a493ed None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:25.847523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6eb11653-097b-4eed-ba1a-3c3aa8a493ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:25.861150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 810/1618] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.865343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d4b2-06f6-4240-bb61-2615adfc5fca None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.876635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d4b2-06f6-4240-bb61-2615adfc5fca None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.881287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:25.881902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:25.882170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:25.882433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:25.883070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:25.885465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:25.916569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 811/1619] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:25.917942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.922794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:25.939958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 809/1620] 10.222.0.23 () {58 vars in 1059 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:25.940959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb11653-097b-4eed-ba1a-3c3aa8a493ed None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:25.952922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb11653-097b-4eed-ba1a-3c3aa8a493ed None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:25.997942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a0ad0-c62f-4bdd-b3d3-36aadff9151c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.002918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.003604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.003941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.004308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.005010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.007096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.014038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:26.021044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 812/1621] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:26.022448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.027285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.027919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.028296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.028646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.029119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.031053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.033880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87998d1-3597-434c-ac64-d5c592b53d78 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.038458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.039076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.039341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.039611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.040261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.042795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.066788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 813/1622] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:26 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:26.067809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.086287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 810/1623] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:25 2022] POST /identity/v3/projects => generated 355 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:26.088565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.191498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9853b2-3ad2-4ab0-a045-6aef053b1ed1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.201039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a82efa-bf74-4d2b-82b3-851cafc0a0a6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.209425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.210241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.210563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.211008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] PATH_INFO: `/v3/projects/bf32c6c197874ff7bbbfdfb1e3d354d8/users/475ff79ea3954154965255915aa45420/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.234213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf32c6c197874ff7bbbfdfb1e3d354d8, user_id=475ff79ea3954154965255915aa45420, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.238246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.263109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 811/1624] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:26 2022] PUT /identity/v3/projects/bf32c6c197874ff7bbbfdfb1e3d354d8/users/475ff79ea3954154965255915aa45420/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:26.263646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.334590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.342673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.343031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.343308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.368216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b8a9e-9461-4810-bbfd-eaca7d07a666 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.387093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.388067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.388547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.389001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.390267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.392528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.408552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 812/1625] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:26.410215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.423827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.425792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:26.426578 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.432703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcc52d-56eb-4f29-b03d-c7e550c16355 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.443134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.444180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.445045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 814/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:26.445446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.445779 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.446653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.447373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.448996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.459115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a32f93c-8676-4e9c-afdd-29cc204e831a service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.466141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.467300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.468061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.468758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.469706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.472119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.483273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 813/1627] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:26 2022] POST /identity/v3/users => generated 284 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:26.483273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.492571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 815/1628] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:26.493726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.501647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5b34e-8d69-41a4-92ea-0712a49f20c5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.506263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55f8e6-6625-429b-b8df-8d5826e884e8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.508690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.512724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.514143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.514892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] PATH_INFO: `/v3/projects/935bc4f749d44c8f8b251f5967e0ba81/users/c26dc937b3dd482c9dcaf81bbee9f4e1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.516211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.518612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.518904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.519183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.519830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.527273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.528983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:26.562489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=935bc4f749d44c8f8b251f5967e0ba81, user_id=c26dc937b3dd482c9dcaf81bbee9f4e1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.569124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 816/1629] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:26.570147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.571259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.596758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b4558-c6e9-490f-bfbd-641dfcd63674 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.609107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 814/1630] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:26 2022] PUT /identity/v3/projects/935bc4f749d44c8f8b251f5967e0ba81/users/c26dc937b3dd482c9dcaf81bbee9f4e1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:26.611766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.741055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.741849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.742174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.742457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] PATH_INFO: `/v3/projects/bf32c6c197874ff7bbbfdfb1e3d354d8/users/475ff79ea3954154965255915aa45420/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.744879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf32c6c197874ff7bbbfdfb1e3d354d8, user_id=475ff79ea3954154965255915aa45420, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.747600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.756109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcedcb7-d989-4054-b26b-de21455afd58 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:26.763045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515391ca-a318-4cb3-880c-9b372ce4b5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:26.763936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515391ca-a318-4cb3-880c-9b372ce4b5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.764277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515391ca-a318-4cb3-880c-9b372ce4b5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.764552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515391ca-a318-4cb3-880c-9b372ce4b5e8 admin admin] PATH_INFO: `/v3/projects/f0e3f90511d54c9b8d0a3188b71d7f1b/users/e5d219103d2d4a65a7cf0890338aa4a8/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.771465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 817/1631] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:26 2022] PUT /identity/v3/projects/bf32c6c197874ff7bbbfdfb1e3d354d8/users/475ff79ea3954154965255915aa45420/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:26.772379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:26.787859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515391ca-a318-4cb3-880c-9b372ce4b5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e3f90511d54c9b8d0a3188b71d7f1b, user_id=e5d219103d2d4a65a7cf0890338aa4a8, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:26.792351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515391ca-a318-4cb3-880c-9b372ce4b5e8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:26.830442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 815/1632] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:26 2022] PUT /identity/v3/projects/f0e3f90511d54c9b8d0a3188b71d7f1b/users/e5d219103d2d4a65a7cf0890338aa4a8/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:26.834486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94f9b-094e-4867-93aa-a2faaf547474 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:26.834785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94f9b-094e-4867-93aa-a2faaf547474 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:26.835064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94f9b-094e-4867-93aa-a2faaf547474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:26.895304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-43a94f9b-094e-4867-93aa-a2faaf547474 None None] MFA Rules not processed for user `475ff79ea3954154965255915aa45420`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:26.896788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-43a94f9b-094e-4867-93aa-a2faaf547474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:26.911106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218fcc00-423c-46e8-9fcc-c32cbe4ed20f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:27.026045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:27.026881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.027338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.027618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.028376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:27.030080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:27.043584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 818/1633] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:27.044674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:27.045601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 816/1634] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:27.047280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36185a-f359-40d4-8d34-aaa38ed5544c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.047792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36185a-f359-40d4-8d34-aaa38ed5544c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.048337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36185a-f359-40d4-8d34-aaa38ed5544c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.051712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 817/1635] 10.222.0.23 () {54 vars in 912 bytes} [Mon Sep 5 17:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:27.052805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36185a-f359-40d4-8d34-aaa38ed5544c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:27.063421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a142457b-bb31-497b-8c79-6195b7f68f61 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:27.069846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ffa3d-035a-4fd8-ab32-9bf2bf9972ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:27.071855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ffa3d-035a-4fd8-ab32-9bf2bf9972ae admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.072566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ffa3d-035a-4fd8-ab32-9bf2bf9972ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.072852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ffa3d-035a-4fd8-ab32-9bf2bf9972ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.073678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ffa3d-035a-4fd8-ab32-9bf2bf9972ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:27.076747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ffa3d-035a-4fd8-ab32-9bf2bf9972ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:27.088731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36185a-f359-40d4-8d34-aaa38ed5544c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:27.099379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 819/1636] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:27.101277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:27.102722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.103101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.104298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] PATH_INFO: `/v3/projects/935bc4f749d44c8f8b251f5967e0ba81/users/c26dc937b3dd482c9dcaf81bbee9f4e1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.106512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77453121-183f-4827-8052-8ac23ad3a0fe None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.106799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77453121-183f-4827-8052-8ac23ad3a0fe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.107641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=935bc4f749d44c8f8b251f5967e0ba81, user_id=c26dc937b3dd482c9dcaf81bbee9f4e1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:27.108702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77453121-183f-4827-8052-8ac23ad3a0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.115128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:27.142328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-77453121-183f-4827-8052-8ac23ad3a0fe None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:27.143833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-77453121-183f-4827-8052-8ac23ad3a0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:27.168176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:27.169248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 818/1637] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:27 2022] PUT /identity/v3/projects/935bc4f749d44c8f8b251f5967e0ba81/users/c26dc937b3dd482c9dcaf81bbee9f4e1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:27.172444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:27.221184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:27.247094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 820/1638] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:27.249014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77453121-183f-4827-8052-8ac23ad3a0fe None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:27.291020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4013-2ceb-43a0-9ad5-c3d1b53739ef admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:27.295107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:27.295868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.296190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.296582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.385916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77453121-183f-4827-8052-8ac23ad3a0fe None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:27.393815 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54403b3a-c4de-49b2-8b6b-ba6cfad78c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:27.395579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54403b3a-c4de-49b2-8b6b-ba6cfad78c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.395863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54403b3a-c4de-49b2-8b6b-ba6cfad78c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.396163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54403b3a-c4de-49b2-8b6b-ba6cfad78c1f admin admin] PATH_INFO: `/v3/projects/f0e3f90511d54c9b8d0a3188b71d7f1b/users/e5d219103d2d4a65a7cf0890338aa4a8/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.398805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54403b3a-c4de-49b2-8b6b-ba6cfad78c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e3f90511d54c9b8d0a3188b71d7f1b, user_id=e5d219103d2d4a65a7cf0890338aa4a8, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:27.401782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54403b3a-c4de-49b2-8b6b-ba6cfad78c1f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:27.416460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:27.417112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 821/1639] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:27 2022] PUT /identity/v3/projects/f0e3f90511d54c9b8d0a3188b71d7f1b/users/e5d219103d2d4a65a7cf0890338aa4a8/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:27.418289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:27.419034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841692-3d46-4a4b-970c-280a8d35f11e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.419499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841692-3d46-4a4b-970c-280a8d35f11e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.419929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841692-3d46-4a4b-970c-280a8d35f11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.436665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 819/1640] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:27.437834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:27.479834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b331e3-c3c1-4421-918c-c848451de283 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:27.490087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b5841692-3d46-4a4b-970c-280a8d35f11e None None] MFA Rules not processed for user `c26dc937b3dd482c9dcaf81bbee9f4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:27.491486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b5841692-3d46-4a4b-970c-280a8d35f11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:27.552685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:27.554805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.554805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.555101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.641087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 822/1641] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:27.644456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1bdb-aa51-4da7-9ecf-707c5cfd216d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.644916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1bdb-aa51-4da7-9ecf-707c5cfd216d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.645482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1bdb-aa51-4da7-9ecf-707c5cfd216d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.688295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:27.690212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:27.703640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 820/1642] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:27.704977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:27.706583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-26fa1bdb-aa51-4da7-9ecf-707c5cfd216d None None] MFA Rules not processed for user `e5d219103d2d4a65a7cf0890338aa4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:27.708548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26fa1bdb-aa51-4da7-9ecf-707c5cfd216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:27.720238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703f0f4-c52e-42b1-b8df-b6a164060e8c service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:27.728204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:27.729723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:27.730347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:27.730899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:27.737738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:27.744764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:27.830723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 823/1643] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:27.830962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:27.830962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:27.843443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 821/1644] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:28.141157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa1bdb-aa51-4da7-9ecf-707c5cfd216d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:28.182976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa1bdb-aa51-4da7-9ecf-707c5cfd216d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:28.193847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:28.194554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.195461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.195753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:28.237681 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:28.239217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:28.270546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 824/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:28.302307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:28.316347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d0053-2c1f-43a5-85ec-7c41bf4b0c4f service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:28.321134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:28.321826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.322158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.322441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:28.336231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:28.348572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:28.377386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 822/1646] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:28.556920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:28.572526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6d4c0-e2f9-4be3-a014-3c52f45ab6f0 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:28.597910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3ae16-10ce-403f-9229-0ee7350481dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:28.598805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3ae16-10ce-403f-9229-0ee7350481dd service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.600760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3ae16-10ce-403f-9229-0ee7350481dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.601449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3ae16-10ce-403f-9229-0ee7350481dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:28.624388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:28.640593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6896f3-80ad-4d2e-acc4-05c4c2e2ac39 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:28.653776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfc798-42c8-40db-8ebd-9cf3c604ba6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:28.656088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfc798-42c8-40db-8ebd-9cf3c604ba6a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.656767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfc798-42c8-40db-8ebd-9cf3c604ba6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.657053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfc798-42c8-40db-8ebd-9cf3c604ba6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:28.806690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3ae16-10ce-403f-9229-0ee7350481dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:28.808535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d3ae16-10ce-403f-9229-0ee7350481dd service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:28.830823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 825/1647] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:28.834187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bc640-a104-4a22-9b9b-e2aa15ac29e4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.834695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bc640-a104-4a22-9b9b-e2aa15ac29e4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.835678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8bc640-a104-4a22-9b9b-e2aa15ac29e4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:28.837289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfc798-42c8-40db-8ebd-9cf3c604ba6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:28.838838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfc798-42c8-40db-8ebd-9cf3c604ba6a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:28.840537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 826/1648] 10.222.0.23 () {54 vars in 962 bytes} [Mon Sep 5 17:01:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:28.846005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029c0b6-0eff-4479-991d-87dfe2434d75 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.846641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029c0b6-0eff-4479-991d-87dfe2434d75 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.847509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029c0b6-0eff-4479-991d-87dfe2434d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:28.863822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 823/1649] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:28.910709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8029c0b6-0eff-4479-991d-87dfe2434d75 None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:28.917735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8029c0b6-0eff-4479-991d-87dfe2434d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:28.942355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 827/1650] 10.222.0.23 () {58 vars in 1059 bytes} [Mon Sep 5 17:01:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:28.948376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099ba19-0d07-43c2-b46c-2a2d0ed0d34b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.948807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099ba19-0d07-43c2-b46c-2a2d0ed0d34b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.949099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099ba19-0d07-43c2-b46c-2a2d0ed0d34b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:28.950881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 824/1651] 10.222.0.23 () {54 vars in 912 bytes} [Mon Sep 5 17:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:28.957822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a272af-7924-4e77-8360-8e272e10bfd1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:28.958419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a272af-7924-4e77-8360-8e272e10bfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:28.958995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a272af-7924-4e77-8360-8e272e10bfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:29.016070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-99a272af-7924-4e77-8360-8e272e10bfd1 None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:29.021302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-99a272af-7924-4e77-8360-8e272e10bfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:29.047917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 828/1652] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:01:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:29.085946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099ba19-0d07-43c2-b46c-2a2d0ed0d34b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:29.113038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099ba19-0d07-43c2-b46c-2a2d0ed0d34b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:29.133332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eeaf62-8da8-4365-9dda-e563f237fc60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:29.134055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeaf62-8da8-4365-9dda-e563f237fc60 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:29.134442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeaf62-8da8-4365-9dda-e563f237fc60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:29.140237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eeaf62-8da8-4365-9dda-e563f237fc60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:29.179190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeaf62-8da8-4365-9dda-e563f237fc60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:29.191422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eeaf62-8da8-4365-9dda-e563f237fc60 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:29.238028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 825/1653] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:36.129516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f2b58-f744-4cf2-b376-8eccb946bbf4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:36.130092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f2b58-f744-4cf2-b376-8eccb946bbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:36.130092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f2b58-f744-4cf2-b376-8eccb946bbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:36.183708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-109f2b58-f744-4cf2-b376-8eccb946bbf4 None None] MFA Rules not processed for user `117a819351b742deb901c081d0a4347b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:36.185900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-109f2b58-f744-4cf2-b376-8eccb946bbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:36.301276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 829/1654] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:36.312167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1acd0-2dec-4362-bed8-5b20fcf9bd7e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:36.312715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1acd0-2dec-4362-bed8-5b20fcf9bd7e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:36.312926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1acd0-2dec-4362-bed8-5b20fcf9bd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:36.341912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c1b1acd0-2dec-4362-bed8-5b20fcf9bd7e None None] MFA Rules not processed for user `117a819351b742deb901c081d0a4347b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:36.343374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c1b1acd0-2dec-4362-bed8-5b20fcf9bd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:36.361376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 826/1655] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:36.402999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f2b58-f744-4cf2-b376-8eccb946bbf4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:36.418310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f2b58-f744-4cf2-b376-8eccb946bbf4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:36.424972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecad51-9959-43f3-906e-93a8a045b933 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:36.425713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecad51-9959-43f3-906e-93a8a045b933 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:36.425977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecad51-9959-43f3-906e-93a8a045b933 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:36.426920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecad51-9959-43f3-906e-93a8a045b933 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:36.437418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecad51-9959-43f3-906e-93a8a045b933 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:36.439120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecad51-9959-43f3-906e-93a8a045b933 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:36.449455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 830/1656] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:36.853315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ff5a0-2c7c-4ca6-98e6-9b0c645bc1f6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:36.853640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ff5a0-2c7c-4ca6-98e6-9b0c645bc1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:36.854004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ff5a0-2c7c-4ca6-98e6-9b0c645bc1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:36.948058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-185ff5a0-2c7c-4ca6-98e6-9b0c645bc1f6 None None] MFA Rules not processed for user `c53ac49fbc054c01a5a1ef41e04bf253`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:36.949561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-185ff5a0-2c7c-4ca6-98e6-9b0c645bc1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:37.111217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c93a1-7e3c-4949-97d4-5e710bdcee48 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:37.111767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c93a1-7e3c-4949-97d4-5e710bdcee48 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:37.112223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c93a1-7e3c-4949-97d4-5e710bdcee48 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:37.114362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 831/1657] 10.222.0.23 () {54 vars in 912 bytes} [Mon Sep 5 17:01:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:37.121929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85607aae-ecaf-4627-95d5-0f5a6af3e854 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:37.122410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85607aae-ecaf-4627-95d5-0f5a6af3e854 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:37.122906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85607aae-ecaf-4627-95d5-0f5a6af3e854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:37.127204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:37.133433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 827/1658] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:37.151078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ff5a0-2c7c-4ca6-98e6-9b0c645bc1f6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:37.167799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:37.218261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ff5a0-2c7c-4ca6-98e6-9b0c645bc1f6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:37.236875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:37.237483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:37.237754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:37.238019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:37.238838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-85607aae-ecaf-4627-95d5-0f5a6af3e854 None None] MFA Rules not processed for user `fcbcd7638cc84cdd87a1eb98f5926425`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:37.240324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-85607aae-ecaf-4627-95d5-0f5a6af3e854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:37.264594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:37.266264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:37.291741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 828/1659] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:37.388646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 832/1660] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:01:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:37.401368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:37.403649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:37.433553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7851d-9740-472b-9957-0bca9ea041df service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:37.440093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbc072-f50c-45d4-ba98-808f7481bb15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:37.441000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc072-f50c-45d4-ba98-808f7481bb15 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:37.441418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc072-f50c-45d4-ba98-808f7481bb15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:37.441827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbc072-f50c-45d4-ba98-808f7481bb15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:37.459633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbc072-f50c-45d4-ba98-808f7481bb15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:37.461130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbc072-f50c-45d4-ba98-808f7481bb15 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:37.499389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 829/1661] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:37.864963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9225-45fe-4c18-b19f-6ca73d2ce576 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:37.865370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9225-45fe-4c18-b19f-6ca73d2ce576 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:37.865800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9225-45fe-4c18-b19f-6ca73d2ce576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:37.918197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-82db9225-45fe-4c18-b19f-6ca73d2ce576 None None] MFA Rules not processed for user `06e60909a8b04008af659620527e3236`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:37.919660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-82db9225-45fe-4c18-b19f-6ca73d2ce576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:38.110188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 833/1662] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:38.110515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fb4f8-6d4f-47f8-b7fa-5c902c96789c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:38.110856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fb4f8-6d4f-47f8-b7fa-5c902c96789c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:38.111134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fb4f8-6d4f-47f8-b7fa-5c902c96789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:38.144869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b65fb4f8-6d4f-47f8-b7fa-5c902c96789c None None] MFA Rules not processed for user `06e60909a8b04008af659620527e3236`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:38.146279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b65fb4f8-6d4f-47f8-b7fa-5c902c96789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:38.167075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 830/1663] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:38.175389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db9225-45fe-4c18-b19f-6ca73d2ce576 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:38.197936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db9225-45fe-4c18-b19f-6ca73d2ce576 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:38.219869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85d427-0142-4d77-b289-f0e68d994c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:38.221276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85d427-0142-4d77-b289-f0e68d994c6e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:38.222749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85d427-0142-4d77-b289-f0e68d994c6e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:38.223025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85d427-0142-4d77-b289-f0e68d994c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:38.249673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85d427-0142-4d77-b289-f0e68d994c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:38.251524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85d427-0142-4d77-b289-f0e68d994c6e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:38.320287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 834/1664] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:38.390529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fb4f8-6d4f-47f8-b7fa-5c902c96789c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:38.592770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fb4f8-6d4f-47f8-b7fa-5c902c96789c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:38.605285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114abdb7-bb7d-4f72-a9d3-0bad70800c18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:38.606240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114abdb7-bb7d-4f72-a9d3-0bad70800c18 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:38.606690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114abdb7-bb7d-4f72-a9d3-0bad70800c18 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:38.607152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114abdb7-bb7d-4f72-a9d3-0bad70800c18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:38.609361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-114abdb7-bb7d-4f72-a9d3-0bad70800c18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:38.631712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114abdb7-bb7d-4f72-a9d3-0bad70800c18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:38.633685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114abdb7-bb7d-4f72-a9d3-0bad70800c18 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:38.647488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 831/1665] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:38.791170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f5d3e-cfb8-46ff-b814-f60b1c945259 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:38.791460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f5d3e-cfb8-46ff-b814-f60b1c945259 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:38.791736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f5d3e-cfb8-46ff-b814-f60b1c945259 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:38.795112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 835/1666] 10.222.0.23 () {54 vars in 912 bytes} [Mon Sep 5 17:01:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:38.800876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745389e2-5a54-442b-98ba-13264489bf3b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:38.801446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745389e2-5a54-442b-98ba-13264489bf3b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:38.801746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745389e2-5a54-442b-98ba-13264489bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:38.836410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-745389e2-5a54-442b-98ba-13264489bf3b None None] MFA Rules not processed for user `fcbcd7638cc84cdd87a1eb98f5926425`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:38.837818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-745389e2-5a54-442b-98ba-13264489bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:38.850956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 832/1667] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:01:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:38.857639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f5d3e-cfb8-46ff-b814-f60b1c945259 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:38.871646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f5d3e-cfb8-46ff-b814-f60b1c945259 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:38.878330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac9470-4ecd-4846-8873-362214297c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:38.879115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac9470-4ecd-4846-8873-362214297c25 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:38.879526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac9470-4ecd-4846-8873-362214297c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:38.879916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac9470-4ecd-4846-8873-362214297c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:38.890069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac9470-4ecd-4846-8873-362214297c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:38.891745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac9470-4ecd-4846-8873-362214297c25 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:38.900997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 836/1668] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:39.643462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb5a61-e397-4cad-bc32-ecc10fb4e8a6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:39.644011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb5a61-e397-4cad-bc32-ecc10fb4e8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:39.644011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb5a61-e397-4cad-bc32-ecc10fb4e8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:39.720465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ddbb5a61-e397-4cad-bc32-ecc10fb4e8a6 None None] MFA Rules not processed for user `06e60909a8b04008af659620527e3236`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:39.723698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ddbb5a61-e397-4cad-bc32-ecc10fb4e8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:39.744501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 833/1669] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:40.611231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:40.611789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:40.611965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:40.651898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] MFA Rules not processed for user `c53ac49fbc054c01a5a1ef41e04bf253`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:40.653506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:40.673371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 837/1670] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:48.640270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ddeb-35de-46b9-b790-411ebc20f3da None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:48.641763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ddeb-35de-46b9-b790-411ebc20f3da None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:48.642186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ddeb-35de-46b9-b790-411ebc20f3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:48.761120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f783ddeb-35de-46b9-b790-411ebc20f3da None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:48.780263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f783ddeb-35de-46b9-b790-411ebc20f3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:48.951905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 834/1671] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:01:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:49.117888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:49.121347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:49.192128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:49.230012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf9f82-1b22-4a10-8d74-515cc6134f96 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:49.243939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:49.244624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:49.245872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:49.246280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:49.258702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:49.262577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:49.275566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 838/1672] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:01:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:01:50.588269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0717e9-dd36-4276-839c-7cc3fcd35a5c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:50.589207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0717e9-dd36-4276-839c-7cc3fcd35a5c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:50.589649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0717e9-dd36-4276-839c-7cc3fcd35a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:50.659455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3a0717e9-dd36-4276-839c-7cc3fcd35a5c None None] MFA Rules not processed for user `fcbcd7638cc84cdd87a1eb98f5926425`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:50.661057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3a0717e9-dd36-4276-839c-7cc3fcd35a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:50.700941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 835/1673] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Sep 5 17:01:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:50.733592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:50.765454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dae2c-df14-45fc-b46c-23ee3384f1ca service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:50.821117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:50.822564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:50.823231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:50.823718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:50.878367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:50.880069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:50.959458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 839/1674] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:51.054050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0717e9-dd36-4276-839c-7cc3fcd35a5c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:51.089851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0717e9-dd36-4276-839c-7cc3fcd35a5c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:51.118260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccff06f-6649-4e1e-965d-7fdab4f31694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:51.119927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccff06f-6649-4e1e-965d-7fdab4f31694 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:51.120640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccff06f-6649-4e1e-965d-7fdab4f31694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:51.121132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccff06f-6649-4e1e-965d-7fdab4f31694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:51.135486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:51.166857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccff06f-6649-4e1e-965d-7fdab4f31694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:51.172881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccff06f-6649-4e1e-965d-7fdab4f31694 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:51.186731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 836/1675] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:01:51.864632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:52.158716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70aa72-820f-4554-a975-e5f85e42868d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:52.174763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:52.183336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:52.184054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:52.185348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] PATH_INFO: `/v3/users/e2d470c0a2624ddcbb015ca02598e5d4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:52.189248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d470c0a2624ddcbb015ca02598e5d4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:52.218334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:52.367882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d470c0a2624ddcbb015ca02598e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:52.393040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d470c0a2624ddcbb015ca02598e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:52.441534 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d470c0a2624ddcbb015ca02598e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:52.454405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d470c0a2624ddcbb015ca02598e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:52.480868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d470c0a2624ddcbb015ca02598e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:52.491908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 840/1676] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:01:51 2022] DELETE /identity/v3/users/e2d470c0a2624ddcbb015ca02598e5d4 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:52.496353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:52.496993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:52.496993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:52.544987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:52.548751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:52.686143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 837/1677] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:52.776699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:52.778604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:52.959449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:53.044509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8bd6c-c5d4-4a54-bef1-43caae9e2872 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:53.056740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c4cd9-31ec-4eb9-823f-0f4d336db6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:53.059250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4cd9-31ec-4eb9-823f-0f4d336db6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:53.060563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4cd9-31ec-4eb9-823f-0f4d336db6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:53.061495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4cd9-31ec-4eb9-823f-0f4d336db6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:53.089098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c4cd9-31ec-4eb9-823f-0f4d336db6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:53.092018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c4cd9-31ec-4eb9-823f-0f4d336db6f2 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:53.136320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 841/1678] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:54.561903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:54.564867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:54.589489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddc938-f79f-4a7a-bdc8-8871aa68ed25 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:54.605323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:54.607684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:54.608224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:54.608809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] PATH_INFO: `/v3/users/212e5af7a7674a16b1ca74bab4da5309` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:54.613635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212e5af7a7674a16b1ca74bab4da5309)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:54.615107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 17:01:54.615107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Sep 05 17:01:54.615107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: Sep 05 17:01:54.616820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:54.786944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212e5af7a7674a16b1ca74bab4da5309'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:54.791531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212e5af7a7674a16b1ca74bab4da5309'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:54.817176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212e5af7a7674a16b1ca74bab4da5309'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:54.884459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212e5af7a7674a16b1ca74bab4da5309'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:54.884459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212e5af7a7674a16b1ca74bab4da5309'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:54.916066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 838/1679] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:01:54 2022] DELETE /identity/v3/users/212e5af7a7674a16b1ca74bab4da5309 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:54.921818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6097-e72a-4669-8f98-37acdf95db81 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:54.922713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6097-e72a-4669-8f98-37acdf95db81 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:54.922713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6097-e72a-4669-8f98-37acdf95db81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:54.988691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-972c6097-e72a-4669-8f98-37acdf95db81 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:54.995794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-972c6097-e72a-4669-8f98-37acdf95db81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:55.228484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 842/1680] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:55.254649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:55.256979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:55.443174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b165-49ad-4234-9886-06517ff17a45 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:55.453711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047874d7-6c8f-467a-8c67-e1d761c56ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:55.454423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047874d7-6c8f-467a-8c67-e1d761c56ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:55.454720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047874d7-6c8f-467a-8c67-e1d761c56ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:55.455035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047874d7-6c8f-467a-8c67-e1d761c56ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:55.484289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047874d7-6c8f-467a-8c67-e1d761c56ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:55.487852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047874d7-6c8f-467a-8c67-e1d761c56ae0 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:55.517599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 839/1681] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:55.659781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be3c5-e7e2-426f-88c9-035837bcae99 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:55.660073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be3c5-e7e2-426f-88c9-035837bcae99 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:55.660388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be3c5-e7e2-426f-88c9-035837bcae99 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:55.661770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 843/1682] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:55.743460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc978-e058-4cde-af08-77b01f482950 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:55.744521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc978-e058-4cde-af08-77b01f482950 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:55.745301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefc978-e058-4cde-af08-77b01f482950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:55.836996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ffefc978-e058-4cde-af08-77b01f482950 None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:55.839311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ffefc978-e058-4cde-af08-77b01f482950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:55.851449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:55.919468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548ce4-c7df-4027-86f7-a54b774bc13a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:55.921538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548ce4-c7df-4027-86f7-a54b774bc13a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:55.922204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2548ce4-c7df-4027-86f7-a54b774bc13a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:55.928984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 844/1683] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:01:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:01:55.950422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2548ce4-c7df-4027-86f7-a54b774bc13a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:55.956439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c2548ce4-c7df-4027-86f7-a54b774bc13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:55.990455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2548ce4-c7df-4027-86f7-a54b774bc13a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:56.001294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:56.012391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.013883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.014757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] PATH_INFO: `/v3/projects/bdc64dc236bf4a87b93a1561ff53293b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.017844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc64dc236bf4a87b93a1561ff53293b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:56.019956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 05 17:01:56.019956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Sep 05 17:01:56.019956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: Sep 05 17:01:56.022157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:56.056057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 840/1684] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:01:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:56.059077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.059328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.060025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.108715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:56.112876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.143889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 841/1685] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:01:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:56.162946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc64dc236bf4a87b93a1561ff53293b', 'request_id': 'req-da707d9f-ef4a-4228-90c4-1dd0418ff766'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:56.181605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc64dc236bf4a87b93a1561ff53293b', 'request_id': 'req-da707d9f-ef4a-4228-90c4-1dd0418ff766'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:56.191525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc64dc236bf4a87b93a1561ff53293b', 'request_id': 'req-da707d9f-ef4a-4228-90c4-1dd0418ff766'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:56.236720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] The token cache is being invalidate because project bdc64dc236bf4a87b93a1561ff53293b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:01:56.237027 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:56.239429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da707d9f-ef4a-4228-90c4-1dd0418ff766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:56.241052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:56.243229 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.247061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 845/1686] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:01:55 2022] DELETE /identity/v3/projects/bdc64dc236bf4a87b93a1561ff53293b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:56.253493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576ad4f-b193-48d9-bc81-c67b4ee528ab None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.253787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576ad4f-b193-48d9-bc81-c67b4ee528ab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.254067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576ad4f-b193-48d9-bc81-c67b4ee528ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.294696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9576ad4f-b193-48d9-bc81-c67b4ee528ab None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:56.297875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9576ad4f-b193-48d9-bc81-c67b4ee528ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.298852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba001e7-a821-4b55-a73f-6ef176a983a4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:56.505570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 846/1687] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:56.514065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.514439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.516412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.569445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:56.572882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:56.573559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.573929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.574700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.589435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.613965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:56.615353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.631478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:56.639173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:56.646351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 847/1688] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:56.646351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:56.650986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.685252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 842/1689] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:01:56.690458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:56.706059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f9e66-bb1e-4171-9bf2-2906c25bf108 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:56.729573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:56.729776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac913d-377a-4c3f-8846-51af3b7bc2cf service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:56.730583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.733203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.733509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.745997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.747122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:56.749864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.751366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.757936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.761562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:01:56.766916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:56.774253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:56.813264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:56.815634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:56.825140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 843/1690] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:01:56.826278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:56.868054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9311dc4-5ee3-4c81-ad92-d438bd40b948 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:56.892288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:56.894095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:56.896824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:56.897203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:56.901353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 848/1691] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:56.911362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:56.918420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:56.920696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:56.922144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:56.979926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257bf6d-1938-4de2-a6f2-ba6fee29ef4e service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:57.008818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:57.009776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:57.010384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:57.010839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] PATH_INFO: `/v3/projects/adfd09010e3446bf998019b95c890156` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:57.021324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adfd09010e3446bf998019b95c890156)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:57.024745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:57.036140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 844/1692] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:01:56 2022] POST /identity/v3/projects => generated 405 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:57.037700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:57.219027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:57.234311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adfd09010e3446bf998019b95c890156', 'request_id': 'req-5f42e233-d36b-46ab-9941-970deb937376'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.254655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adfd09010e3446bf998019b95c890156', 'request_id': 'req-5f42e233-d36b-46ab-9941-970deb937376'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.265746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adfd09010e3446bf998019b95c890156', 'request_id': 'req-5f42e233-d36b-46ab-9941-970deb937376'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.271676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5096f35-3e3f-43eb-91d2-d3d6063dab28 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:57.298787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] The token cache is being invalidate because project adfd09010e3446bf998019b95c890156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:01:57.299225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.299986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.307544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 849/1693] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:01:56 2022] DELETE /identity/v3/projects/adfd09010e3446bf998019b95c890156 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:57.309052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:57.311397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:57.498483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:57.499812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:57.500377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:57.501483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:57.503580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:57.511149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:57.542069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42e233-d36b-46ab-9941-970deb937376 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:57.559983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:57.564305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:57.564940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:57.565624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] PATH_INFO: `/v3/users/c26dc937b3dd482c9dcaf81bbee9f4e1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:57.569091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 845/1694] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:01:57 2022] POST /identity/v3/users => generated 303 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:01:57.570320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26dc937b3dd482c9dcaf81bbee9f4e1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:57.580446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:57.583186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:57.586715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:57.621358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840f7b6-7cb4-4010-8810-65935c9c19c2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:57.634717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:57.640419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:57.640901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:57.641168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:57.642449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:57.645255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:57.674185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 846/1695] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:57.679003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:57.701003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26dc937b3dd482c9dcaf81bbee9f4e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.705880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97802f54-130d-4924-9154-f42af6bd99b6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:57.714472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26dc937b3dd482c9dcaf81bbee9f4e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.716331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:57.717864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:57.718138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:57.718405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:57.721153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:57.725087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:57.740125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:57.743471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26dc937b3dd482c9dcaf81bbee9f4e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.745025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26dc937b3dd482c9dcaf81bbee9f4e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.755893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 847/1696] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:01:57.757928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cf205159-2fd4-48d0-988d-03a61724bd0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26dc937b3dd482c9dcaf81bbee9f4e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:57.759090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:57.771645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:57.781446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 850/1697] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:01:57 2022] DELETE /identity/v3/users/c26dc937b3dd482c9dcaf81bbee9f4e1 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:57.802387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702ae3d-dacc-47ed-b66b-167d08fa9d6a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:57.933093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:57.933829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:57.934443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:57.934970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] PATH_INFO: `/v3/projects/6268f00dde9c4e2993cbce95f65c9478/users/b902371c225441bf8c913979996fe262/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:58.006881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6268f00dde9c4e2993cbce95f65c9478, user_id=b902371c225441bf8c913979996fe262, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:58.017653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:58.052018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 848/1698] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:01:57 2022] PUT /identity/v3/projects/6268f00dde9c4e2993cbce95f65c9478/users/b902371c225441bf8c913979996fe262/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:58.053821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:58.054266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:58.054430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:58.113369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:58.114637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:58.335040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 851/1699] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:58.342667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:58.344930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:58.445641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:58.448674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:58.469415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392669a-18a5-413e-ab2c-5e15cdd06494 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:58.475497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:58.479881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:58.480990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:58.481573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] PATH_INFO: `/v3/users/c7c5ae21a0ef42a49c14d2665a2d5752` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:58.482962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c5ae21a0ef42a49c14d2665a2d5752)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:58.485111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:58.501189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:58.527193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f058a-06bc-465e-b7d3-f295e67894cf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:58.534876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af01cff-6c08-4fdf-8b72-a3dd1288fabd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:58.538556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af01cff-6c08-4fdf-8b72-a3dd1288fabd service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:58.538864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af01cff-6c08-4fdf-8b72-a3dd1288fabd service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:58.539140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af01cff-6c08-4fdf-8b72-a3dd1288fabd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:58.558754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af01cff-6c08-4fdf-8b72-a3dd1288fabd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:58.560227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af01cff-6c08-4fdf-8b72-a3dd1288fabd service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:58.566441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c5ae21a0ef42a49c14d2665a2d5752'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:58.581010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c5ae21a0ef42a49c14d2665a2d5752'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:58.591711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 849/1700] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:01:58.599636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c5ae21a0ef42a49c14d2665a2d5752'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:58.600454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c5ae21a0ef42a49c14d2665a2d5752'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:58.633364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c5ae21a0ef42a49c14d2665a2d5752'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:01:58.663475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 852/1701] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:01:58 2022] DELETE /identity/v3/users/c7c5ae21a0ef42a49c14d2665a2d5752 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:01:58.665952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:58.666300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:58.666589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:58.711975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:01:58.712793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:58.860989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 850/1702] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:01:58.877522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:01:58.879612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:01:59.110675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7aa88-8d82-44d7-a3d2-abb992f5e945 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:01:59.121918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1d9c6-4db3-417d-91bf-6a21befaa5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:01:59.125651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1d9c6-4db3-417d-91bf-6a21befaa5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:01:59.126281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1d9c6-4db3-417d-91bf-6a21befaa5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:01:59.126821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1d9c6-4db3-417d-91bf-6a21befaa5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:01:59.151066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1d9c6-4db3-417d-91bf-6a21befaa5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:01:59.157527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1d9c6-4db3-417d-91bf-6a21befaa5d7 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:01:59.170862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:01:59.178857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 853/1703] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:00.104452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:00.106039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:00.129425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8443d-af16-4c89-9046-fbfc27f8a10c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:00.137111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:00.138585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:00.139031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:00.140003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] PATH_INFO: `/v3/projects/935bc4f749d44c8f8b251f5967e0ba81` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:00.143366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935bc4f749d44c8f8b251f5967e0ba81)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:00.146496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:00.305667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935bc4f749d44c8f8b251f5967e0ba81', 'request_id': 'req-7bd048e3-7789-4b91-91ed-dc5f731c19d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:00.330784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935bc4f749d44c8f8b251f5967e0ba81', 'request_id': 'req-7bd048e3-7789-4b91-91ed-dc5f731c19d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:00.351819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935bc4f749d44c8f8b251f5967e0ba81', 'request_id': 'req-7bd048e3-7789-4b91-91ed-dc5f731c19d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:00.398121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] The token cache is being invalidate because project 935bc4f749d44c8f8b251f5967e0ba81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:00.398544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:00.401522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7bd048e3-7789-4b91-91ed-dc5f731c19d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:00.405591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 851/1704] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:00 2022] DELETE /identity/v3/projects/935bc4f749d44c8f8b251f5967e0ba81 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:00.413049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72ca9a-013a-413b-89ca-8eb5645b69fd None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:00.413626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72ca9a-013a-413b-89ca-8eb5645b69fd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:00.413626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72ca9a-013a-413b-89ca-8eb5645b69fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:00.475730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9a72ca9a-013a-413b-89ca-8eb5645b69fd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:00.477970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a72ca9a-013a-413b-89ca-8eb5645b69fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:00.599806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 854/1705] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:00.603616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600b652-4008-4d97-bc9a-78468a794151 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:00.604227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600b652-4008-4d97-bc9a-78468a794151 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:00.607276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600b652-4008-4d97-bc9a-78468a794151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:00.657778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b600b652-4008-4d97-bc9a-78468a794151 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:00.659736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b600b652-4008-4d97-bc9a-78468a794151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:00.675976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 852/1706] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:00.680713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72ca9a-013a-413b-89ca-8eb5645b69fd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:00.702819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72ca9a-013a-413b-89ca-8eb5645b69fd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:00.710074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:00.711232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:00.711826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:00.712238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:00.713008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:00.713470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:00.715635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:00.732458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 855/1707] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:00.736723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600b652-4008-4d97-bc9a-78468a794151 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:00.752806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600b652-4008-4d97-bc9a-78468a794151 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:00.779963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:00.780637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:00.780920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:00.781194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:00.781843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:00.783800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:00.890578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 853/1708] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:00 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:00.891311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:00.912717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcba300-c0fa-41c7-a07d-a9d86cc75c88 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:01.117732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:01.122095 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.122888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.123289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.126182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:01.132862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:01.225806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 856/1709] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:00 2022] POST /identity/v3/users => generated 288 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:01.232134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:01.269159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd7902-cbdd-4509-8b3e-f2b3d55b03f4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:01.270434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:01.271688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.272262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.272701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.279651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:01.283545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:01.319910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 854/1710] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:01.325510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:01.354708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c67061-d51a-4c26-93cd-6570f5b00903 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:01.365575 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:01.366363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.366970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.367674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.368521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:01.370577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:01.409709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 857/1711] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:01.413536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:01.446196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6daed-c5e8-4e17-9c64-2934a17b91a5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:01.474998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6959264-ce55-401c-9c7c-12e0e821aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:01.476105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959264-ce55-401c-9c7c-12e0e821aee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.476499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959264-ce55-401c-9c7c-12e0e821aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.477140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959264-ce55-401c-9c7c-12e0e821aee0 admin admin] PATH_INFO: `/v3/projects/33921d60041e46b89dbfd9e110ce17a5/users/c252f69aaf234d018ac99edefa3246d7/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.523436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6959264-ce55-401c-9c7c-12e0e821aee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33921d60041e46b89dbfd9e110ce17a5, user_id=c252f69aaf234d018ac99edefa3246d7, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:01.528494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6959264-ce55-401c-9c7c-12e0e821aee0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:01.547594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 855/1712] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:01 2022] PUT /identity/v3/projects/33921d60041e46b89dbfd9e110ce17a5/users/c252f69aaf234d018ac99edefa3246d7/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:01.556430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:01.558675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:01.565609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b8e4e-1051-4ccd-9da7-e8fe86e7a77b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.565896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b8e4e-1051-4ccd-9da7-e8fe86e7a77b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.566170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b8e4e-1051-4ccd-9da7-e8fe86e7a77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.576104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7027cf-25f3-4a6d-8299-369572c2dbcc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:01.594161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:01.625764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-644b8e4e-1051-4ccd-9da7-e8fe86e7a77b None None] MFA Rules not processed for user `c252f69aaf234d018ac99edefa3246d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:01.627053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-644b8e4e-1051-4ccd-9da7-e8fe86e7a77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:01.668062 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:01.675984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.676568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.676941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] PATH_INFO: `/v3/projects/68ace03f2433467299f56d5392dff17e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.678813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ace03f2433467299f56d5392dff17e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:01.680987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:01.705232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:01.739092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 856/1713] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:01.743363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b8e4e-1051-4ccd-9da7-e8fe86e7a77b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:01.774495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b8e4e-1051-4ccd-9da7-e8fe86e7a77b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:01.788109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ace03f2433467299f56d5392dff17e', 'request_id': 'req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:01.798220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ace03f2433467299f56d5392dff17e', 'request_id': 'req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:01.804590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ace03f2433467299f56d5392dff17e', 'request_id': 'req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:01.834550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] The token cache is being invalidate because project 68ace03f2433467299f56d5392dff17e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:01.837087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:01.838066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28f59a64-c8b5-4fc1-9aed-a14b4913d0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:01.840081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 858/1714] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:01 2022] DELETE /identity/v3/projects/68ace03f2433467299f56d5392dff17e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:01.841957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.842421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.842792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.879417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] MFA Rules not processed for user `b902371c225441bf8c913979996fe262`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:01.880704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:01.882422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80286ede-2fec-41a6-9d55-eb14c1c990df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:01.883463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286ede-2fec-41a6-9d55-eb14c1c990df admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.883973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286ede-2fec-41a6-9d55-eb14c1c990df admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.884612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80286ede-2fec-41a6-9d55-eb14c1c990df admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.885606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80286ede-2fec-41a6-9d55-eb14c1c990df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:01.888256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80286ede-2fec-41a6-9d55-eb14c1c990df admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:01.928745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 857/1715] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:01 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:01.930580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:01.935818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:01.936135 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:01.958804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:01.960198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:01.963940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 859/1716] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:01 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:01.965835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:01.967246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:01.984966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4d886-2596-442e-b51d-6ffad52083a1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.058195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 858/1717] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:02.059051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.060413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.068236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ca4b0-f15a-4e99-9b9b-7fef6259bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.068806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ca4b0-f15a-4e99-9b9b-7fef6259bd64 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.069091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ca4b0-f15a-4e99-9b9b-7fef6259bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.069445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ca4b0-f15a-4e99-9b9b-7fef6259bd64 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.070131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ca4b0-f15a-4e99-9b9b-7fef6259bd64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.072321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ca4b0-f15a-4e99-9b9b-7fef6259bd64 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.076908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6ab5-a865-4282-90a6-08653bbeff3e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.081685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.082363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.082694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.082983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.083623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.085714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.096947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 860/1718] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:01 2022] POST /identity/v3/users => generated 295 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:02.098065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a983-f033-468c-bb20-64a30601e2d8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.098344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a983-f033-468c-bb20-64a30601e2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.098612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a983-f033-468c-bb20-64a30601e2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.116264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:02.120538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f471a983-f033-468c-bb20-64a30601e2d8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:02.122194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f471a983-f033-468c-bb20-64a30601e2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.125596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 859/1719] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:02 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:02.126526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.134406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:02.199236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2c653-6465-4f2f-a4a3-321b3ac44f3c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.201325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 861/1720] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:02.202065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471a983-f033-468c-bb20-64a30601e2d8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.204204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.205083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.205366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.205641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.206379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.208489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.214407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471a983-f033-468c-bb20-64a30601e2d8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.220374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.220976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 860/1721] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:02.221093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.221284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.221527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.221831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.222155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.224421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.225860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.240464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bc89d-137d-49d9-adee-1defd4c4d325 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.246508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.247306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.247459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.247704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] PATH_INFO: `/v3/users/a78f0df187e5487385b5cf6579704d24` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.248871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78f0df187e5487385b5cf6579704d24)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.250809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.255674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 862/1722] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:02 2022] POST /identity/v3/users => generated 308 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:02.255674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.266673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a3579-5656-4559-917b-31dc53c458e9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.273882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.275274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.276021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.276021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.277686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:02.277967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.279981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.299506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 863/1723] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:02.300460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.325411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea8ebe-0b32-4ab6-a1ab-fec2b57b3058 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.334171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.335249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.335692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.336099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] PATH_INFO: `/v3/projects/14be4cc730ca40f683bd64865cf8236d/users/2c0e4fb01b2a4c5fbbceb080b2551c76/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.342977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78f0df187e5487385b5cf6579704d24'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:02.343859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78f0df187e5487385b5cf6579704d24'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:02.357305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14be4cc730ca40f683bd64865cf8236d, user_id=2c0e4fb01b2a4c5fbbceb080b2551c76, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.360278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.361513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78f0df187e5487385b5cf6579704d24'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:02.379572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 864/1724] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:02 2022] PUT /identity/v3/projects/14be4cc730ca40f683bd64865cf8236d/users/2c0e4fb01b2a4c5fbbceb080b2551c76/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:02.380565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.384772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.385866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78f0df187e5487385b5cf6579704d24'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:02.386605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78f0df187e5487385b5cf6579704d24'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:02.400362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 861/1725] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:02 2022] DELETE /identity/v3/users/a78f0df187e5487385b5cf6579704d24 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:02.401199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.402560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.471683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9a115-0715-4334-b9cf-76f6c7623107 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.482536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.483368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.483809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.484254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.485325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.487096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.488857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca4ec0-b948-4d38-80bb-4ff153a0a507 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.494107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69d6d6-f45f-4413-989d-522e4ecbe63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.494733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69d6d6-f45f-4413-989d-522e4ecbe63f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.495027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69d6d6-f45f-4413-989d-522e4ecbe63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.495282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69d6d6-f45f-4413-989d-522e4ecbe63f admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.495925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69d6d6-f45f-4413-989d-522e4ecbe63f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.501432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69d6d6-f45f-4413-989d-522e4ecbe63f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.501598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 865/1726] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:02.501598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.502675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.515620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0df518-8ccb-42c1-917c-1ed765d045f4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.516064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:02.520548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.521161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.521440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.521711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.522363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.524525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.532426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 862/1727] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:02 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:02.534421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb886d2-8ed4-42af-9ddd-be853fbdc3dc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.534808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb886d2-8ed4-42af-9ddd-be853fbdc3dc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.535087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb886d2-8ed4-42af-9ddd-be853fbdc3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.538048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 866/1728] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:02.538724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.550522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154cc3f-be16-4532-b3f6-dd74fa47c163 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.557507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bcb886d2-8ed4-42af-9ddd-be853fbdc3dc None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:02.558726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bcb886d2-8ed4-42af-9ddd-be853fbdc3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.596396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:02.612483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.613173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.613458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.613767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] PATH_INFO: `/v3/projects/4b93ae091c9747b997baecc848881c8b/users/fa9582012dde41c8878bd23f37ffe8ec/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.620751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 863/1729] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:02.624099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb886d2-8ed4-42af-9ddd-be853fbdc3dc None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.630589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b93ae091c9747b997baecc848881c8b, user_id=fa9582012dde41c8878bd23f37ffe8ec, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.631619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb886d2-8ed4-42af-9ddd-be853fbdc3dc None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.633376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.636772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.637417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.637697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.638058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.638765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.640984 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.648599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 867/1730] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:02 2022] PUT /identity/v3/projects/4b93ae091c9747b997baecc848881c8b/users/fa9582012dde41c8878bd23f37ffe8ec/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:02.649502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.664542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 864/1731] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:02 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:02.668503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.670137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.712621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1ce79-db63-4abb-bb8d-da219d9ae675 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.718099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.718733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.719137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.719514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] PATH_INFO: `/v3/projects/14be4cc730ca40f683bd64865cf8236d/users/2c0e4fb01b2a4c5fbbceb080b2551c76/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.721598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] RBAC: Authorizing `identity:create_grant(project_id=14be4cc730ca40f683bd64865cf8236d, user_id=2c0e4fb01b2a4c5fbbceb080b2551c76, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.724552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.729378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd52cf-2b9c-4ae8-83a3-b4efd2177f77 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.740699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 868/1732] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:02 2022] PUT /identity/v3/projects/14be4cc730ca40f683bd64865cf8236d/users/2c0e4fb01b2a4c5fbbceb080b2551c76/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:02.741503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.786132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.787151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.787547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.787886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.798243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.799710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.809700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 865/1733] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:02.810784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.820704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3b4a8-5b6a-4abb-9f6c-33bbffc4593d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.825980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3f56d-0953-44d5-ad01-e0114cde77b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.827244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f56d-0953-44d5-ad01-e0114cde77b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.827244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f56d-0953-44d5-ad01-e0114cde77b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.827802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f56d-0953-44d5-ad01-e0114cde77b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.835271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3f56d-0953-44d5-ad01-e0114cde77b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.837242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5c1dc-557a-4dfa-9b97-6c368ef7896c service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.837591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3f56d-0953-44d5-ad01-e0114cde77b3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.849965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.850920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.851377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.851959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.853188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.855210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.860237 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 869/1734] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:02.862592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.862866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.865046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.869776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 866/1735] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:02.874090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.888803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00317c91-1fa3-4dd8-a1a4-493b83249b46 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:02.896120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:02.897326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:02.897900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:02.898382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] PATH_INFO: `/v3/projects/4b93ae091c9747b997baecc848881c8b/users/fa9582012dde41c8878bd23f37ffe8ec/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:02.901462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b93ae091c9747b997baecc848881c8b, user_id=fa9582012dde41c8878bd23f37ffe8ec, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:02.905196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:02.906569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] MFA Rules not processed for user `2c0e4fb01b2a4c5fbbceb080b2551c76`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:02.907862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:02.922730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 867/1736] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:02 2022] PUT /identity/v3/projects/4b93ae091c9747b997baecc848881c8b/users/fa9582012dde41c8878bd23f37ffe8ec/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:02.924234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:02.960642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:02.972945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:03.014830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 870/1737] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:03.024727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.027070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:03.039031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2a93e-5195-415e-9c34-718e4720434e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.047394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.048031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.048360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.048634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.049287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.051121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.066285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 868/1738] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:03.067626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.083252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db57918-a4a0-413a-ab30-5f6b90be909d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.089730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89630d08-ee3a-480a-9b55-4ea51e3e9082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.090417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630d08-ee3a-480a-9b55-4ea51e3e9082 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.090702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630d08-ee3a-480a-9b55-4ea51e3e9082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.091326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89630d08-ee3a-480a-9b55-4ea51e3e9082 admin admin] PATH_INFO: `/v3/projects/a0928db1d7854f1882d18f3830c57b24/users/86585549f638415ebdee9b7f6438a1fe/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.111145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89630d08-ee3a-480a-9b55-4ea51e3e9082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0928db1d7854f1882d18f3830c57b24, user_id=86585549f638415ebdee9b7f6438a1fe, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.114305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89630d08-ee3a-480a-9b55-4ea51e3e9082 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.120235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbcb1d1-f5ad-458b-904c-19fa75ba0290 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.140468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 869/1739] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:03 2022] PUT /identity/v3/projects/a0928db1d7854f1882d18f3830c57b24/users/86585549f638415ebdee9b7f6438a1fe/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:03.154203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.154708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.155190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.180371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.181080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.181434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.181772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.183827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:03.192033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] MFA Rules not processed for user `86585549f638415ebdee9b7f6438a1fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:03.193796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:03.285890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 870/1740] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:03.286494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.289259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:03.289788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.292181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.304618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 871/1741] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:03.305616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.318459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065118a-372f-4192-9d2d-145094b8a004 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.395011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:03.404723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.405493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.405870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.406244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.407473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.411242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.429417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4b455-484d-4f1f-8924-04fb4d4129a0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.445243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.446302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.446657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.447237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] PATH_INFO: `/v3/projects/069563a168314ecea0dc0d87766a62ce` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.448768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069563a168314ecea0dc0d87766a62ce)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.451679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.464663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 872/1742] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:03 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:03.465920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.488406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:03.572452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069563a168314ecea0dc0d87766a62ce', 'request_id': 'req-13d0000a-dd16-467f-a816-0bcf6038cbd3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.587627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069563a168314ecea0dc0d87766a62ce', 'request_id': 'req-13d0000a-dd16-467f-a816-0bcf6038cbd3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.608030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069563a168314ecea0dc0d87766a62ce', 'request_id': 'req-13d0000a-dd16-467f-a816-0bcf6038cbd3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.612934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b0b3-a0e3-40a0-8c8e-dc8ab416a889 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.640464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] The token cache is being invalidate because project 069563a168314ecea0dc0d87766a62ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:03.641408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.647922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.649810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 871/1743] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:03 2022] DELETE /identity/v3/projects/069563a168314ecea0dc0d87766a62ce => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:03.654087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.659199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:03.702932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.704695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.705091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.705369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.707626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.710900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.763895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 873/1744] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:03 2022] POST /identity/v3/users => generated 288 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:03.768047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.771694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:03.783571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0000a-dd16-467f-a816-0bcf6038cbd3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.789574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.791394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.791681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.791994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] PATH_INFO: `/v3/users/e5d219103d2d4a65a7cf0890338aa4a8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.794914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5d219103d2d4a65a7cf0890338aa4a8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.797704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.799214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc706de-e05f-4cb4-8263-6c7778a7d79d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.813715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.814746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.815443 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.815862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.818149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.820225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.840213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 874/1745] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:03.849239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.866989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c9cf3-5f2f-4819-9dc4-61862c40212b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:03.881420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:03.884730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:03.885459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:03.885946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] PATH_INFO: `/v3/projects/43452046c9d54a53a8efed3ccf0fd677/users/cc0f265709944034b8a8b3854f842074/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:03.894738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5d219103d2d4a65a7cf0890338aa4a8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.897189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5d219103d2d4a65a7cf0890338aa4a8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.916316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=43452046c9d54a53a8efed3ccf0fd677, user_id=cc0f265709944034b8a8b3854f842074, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:03.919866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:03.920018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5d219103d2d4a65a7cf0890338aa4a8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.950960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5d219103d2d4a65a7cf0890338aa4a8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.951554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 875/1746] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:03 2022] PUT /identity/v3/projects/43452046c9d54a53a8efed3ccf0fd677/users/cc0f265709944034b8a8b3854f842074/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:03.951689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5d219103d2d4a65a7cf0890338aa4a8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:03.956286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:03.960697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:03.982406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 872/1747] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:03 2022] DELETE /identity/v3/users/e5d219103d2d4a65a7cf0890338aa4a8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:04.061265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:04.086479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ad6c4-6e89-4e82-95bf-35790ba809ca admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:04.156886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7db61a-fb08-4a53-920d-569c2954e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:04.158069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db61a-fb08-4a53-920d-569c2954e45e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:04.158478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db61a-fb08-4a53-920d-569c2954e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:04.158777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7db61a-fb08-4a53-920d-569c2954e45e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:04.159431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7db61a-fb08-4a53-920d-569c2954e45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:04.161133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7db61a-fb08-4a53-920d-569c2954e45e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:04.172121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 876/1748] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:04.176428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:04.186825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16803e4-769a-4aef-ab6c-3a9ce17b11b9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:04.191838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:04.192087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:04.192399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:04.192685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] PATH_INFO: `/v3/projects/43452046c9d54a53a8efed3ccf0fd677/users/cc0f265709944034b8a8b3854f842074/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:04.194558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43452046c9d54a53a8efed3ccf0fd677, user_id=cc0f265709944034b8a8b3854f842074, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:04.197484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:04.208196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 873/1749] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:04 2022] PUT /identity/v3/projects/43452046c9d54a53a8efed3ccf0fd677/users/cc0f265709944034b8a8b3854f842074/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:04.219142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:04.219458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:04.219734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:04.249608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] MFA Rules not processed for user `cc0f265709944034b8a8b3854f842074`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:04.250884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:04.344348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 877/1750] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:04.355753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:04.359090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:04.372847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:04.441170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4002ae-51d2-4d6e-bee6-8e76912e3bb3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:04.444936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e2046-3e43-4cb8-9533-d463fa992715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:04.445698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e2046-3e43-4cb8-9533-d463fa992715 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:04.446043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e2046-3e43-4cb8-9533-d463fa992715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:04.446320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e2046-3e43-4cb8-9533-d463fa992715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:04.456677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e2046-3e43-4cb8-9533-d463fa992715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:04.458099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e2046-3e43-4cb8-9533-d463fa992715 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:04.469315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 874/1751] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:04.724593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:04.726371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:04.813012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2d041-269b-4000-9695-13874bf7ce03 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:04.813012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2d041-269b-4000-9695-13874bf7ce03 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:04.813012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2d041-269b-4000-9695-13874bf7ce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:04.861806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-20a2d041-269b-4000-9695-13874bf7ce03 None None] MFA Rules not processed for user `fa9582012dde41c8878bd23f37ffe8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:04.863099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-20a2d041-269b-4000-9695-13874bf7ce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:04.865233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411c5ce-ee8a-4bbd-b8f5-0d409e4a1574 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:04.871574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:04.873397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:04.873811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:04.874365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] PATH_INFO: `/v3/projects/f0e3f90511d54c9b8d0a3188b71d7f1b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:04.876625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e3f90511d54c9b8d0a3188b71d7f1b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:04.879548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:04.901343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:04.960187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e3f90511d54c9b8d0a3188b71d7f1b', 'request_id': 'req-d7227b06-ec32-4858-a1bc-ae61c274d1d2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:04.967873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e3f90511d54c9b8d0a3188b71d7f1b', 'request_id': 'req-d7227b06-ec32-4858-a1bc-ae61c274d1d2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:04.974883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e3f90511d54c9b8d0a3188b71d7f1b', 'request_id': 'req-d7227b06-ec32-4858-a1bc-ae61c274d1d2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:04.998345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] The token cache is being invalidate because project f0e3f90511d54c9b8d0a3188b71d7f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:04.998345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:04.999010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d7227b06-ec32-4858-a1bc-ae61c274d1d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:05.001059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 878/1752] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:04 2022] DELETE /identity/v3/projects/f0e3f90511d54c9b8d0a3188b71d7f1b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:05.002213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 875/1753] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:04 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:05.012896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e81480-a325-4181-8467-284928b3df7f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.013264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e81480-a325-4181-8467-284928b3df7f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.013630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e81480-a325-4181-8467-284928b3df7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.052564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-10e81480-a325-4181-8467-284928b3df7f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:05.053796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-10e81480-a325-4181-8467-284928b3df7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:05.127982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2d041-269b-4000-9695-13874bf7ce03 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.141477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2d041-269b-4000-9695-13874bf7ce03 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.146355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.147016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.147477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.147833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] PATH_INFO: `/v3/users/86585549f638415ebdee9b7f6438a1fe` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.148259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 879/1754] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:05.156108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86585549f638415ebdee9b7f6438a1fe)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.156315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242214db-ba34-46a0-a054-b449a4be52a8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.156579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242214db-ba34-46a0-a054-b449a4be52a8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.156879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242214db-ba34-46a0-a054-b449a4be52a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.158977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.181747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-242214db-ba34-46a0-a054-b449a4be52a8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:05.183772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-242214db-ba34-46a0-a054-b449a4be52a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:05.196203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 880/1755] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:05.199423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242214db-ba34-46a0-a054-b449a4be52a8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.210574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242214db-ba34-46a0-a054-b449a4be52a8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.216319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.217051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.217390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.217730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.218476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:05.219499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.221411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.223943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86585549f638415ebdee9b7f6438a1fe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:05.224700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86585549f638415ebdee9b7f6438a1fe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:05.234852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86585549f638415ebdee9b7f6438a1fe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:05.236670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 881/1756] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:05.242538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.245719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:05.252047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86585549f638415ebdee9b7f6438a1fe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:05.255214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9de7698b-c6b7-490f-84a7-7d0ca71955c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86585549f638415ebdee9b7f6438a1fe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:05.275260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 876/1757] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:05 2022] DELETE /identity/v3/users/86585549f638415ebdee9b7f6438a1fe => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:05.278219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daaa24-ec88-4393-b0e2-9b7fc866eacc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.280168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.280459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.281109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.313892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:05.313892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:05.317078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:05.389222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.390119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.390580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.391031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.392046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.394984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.403962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 877/1758] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:05.425723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.427442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:05.448050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 882/1759] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/projects => generated 367 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:05.452265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.464985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:05.506570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c34056c-f4e2-48f9-a9e5-babfe396835c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.540063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbbb23-6916-486c-b560-afdef780901c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.545486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.546117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.546396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.546667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.547308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.549441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.565525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.567599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.567878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.568253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.579776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 883/1760] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:05.580421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.582029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.585387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.592303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 878/1761] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:05.597327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131f10f-b29e-4ca5-b039-73c78c61eee4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.603616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.604296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.604582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.604855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.605493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.607117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.618142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 884/1762] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:05.622212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.631944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7477f-d4a0-4391-b047-2e72aa20d8fa service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.636970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cc953-7607-489b-ab9f-ebbf63ecee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.637816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cc953-7607-489b-ab9f-ebbf63ecee93 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.638094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cc953-7607-489b-ab9f-ebbf63ecee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.638367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cc953-7607-489b-ab9f-ebbf63ecee93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.639004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cc953-7607-489b-ab9f-ebbf63ecee93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.640709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cc953-7607-489b-ab9f-ebbf63ecee93 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.651241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 879/1763] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:05.655416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.665857 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db6601-b017-49ee-afc9-bf89f3dfb0e4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.670416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.671067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.671343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.671614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] PATH_INFO: `/v3/projects/690a4d84f91144ad8bd5a9f8f51b8449/users/a4ee9da54301496e9d7f383036e1c22c/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.687791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] RBAC: Authorizing `identity:create_grant(project_id=690a4d84f91144ad8bd5a9f8f51b8449, user_id=a4ee9da54301496e9d7f383036e1c22c, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.690505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.705034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 885/1764] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:05 2022] PUT /identity/v3/projects/690a4d84f91144ad8bd5a9f8f51b8449/users/a4ee9da54301496e9d7f383036e1c22c/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:05.715475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81ac3-0370-4f9d-8a6b-34989c30d6cc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.715766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81ac3-0370-4f9d-8a6b-34989c30d6cc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.716044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b81ac3-0370-4f9d-8a6b-34989c30d6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.746657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-18b81ac3-0370-4f9d-8a6b-34989c30d6cc None None] MFA Rules not processed for user `a4ee9da54301496e9d7f383036e1c22c`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:05.747875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-18b81ac3-0370-4f9d-8a6b-34989c30d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:05.794273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:05.817852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 880/1765] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:05.822653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.889387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f366e9-0ba3-4614-8b6b-5279849a337f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:05.893058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0027-148b-4962-b37b-373b012c03cb None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.893514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0027-148b-4962-b37b-373b012c03cb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.893850 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0027-148b-4962-b37b-373b012c03cb None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.894314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:05.894943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.895228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.895723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.895843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 881/1766] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:02:05.896171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:05.898121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:05.929026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b551c-130a-4e29-915e-040ac9c03f43 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:05.929595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b551c-130a-4e29-915e-040ac9c03f43 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:05.930648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b551c-130a-4e29-915e-040ac9c03f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:05.936215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 886/1767] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:05.937406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:05.958953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-751b551c-130a-4e29-915e-040ac9c03f43 None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:05.960207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-751b551c-130a-4e29-915e-040ac9c03f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:06.006541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:06.026038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0587e-b0b5-433f-9aca-935a8e18301b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.031827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27272ea-f2fd-4756-bd17-e6a5c9a33f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.032642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27272ea-f2fd-4756-bd17-e6a5c9a33f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.032928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27272ea-f2fd-4756-bd17-e6a5c9a33f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.033201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27272ea-f2fd-4756-bd17-e6a5c9a33f28 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.033900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27272ea-f2fd-4756-bd17-e6a5c9a33f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.036356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27272ea-f2fd-4756-bd17-e6a5c9a33f28 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.059609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 882/1768] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:06.067889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 887/1769] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:05 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:06.071840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b551c-130a-4e29-915e-040ac9c03f43 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.083140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b551c-130a-4e29-915e-040ac9c03f43 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.088810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.089997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.090284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.090641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.091475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.095377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.101476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9f87-bd43-477c-9470-c99b5a8db543 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.101800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9f87-bd43-477c-9470-c99b5a8db543 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.102069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9f87-bd43-477c-9470-c99b5a8db543 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.103802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 888/1770] 10.222.0.23 () {54 vars in 961 bytes} [Mon Sep 5 17:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 17:02:06.109294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baacd0-3b87-4518-8321-3d36fb8c3c97 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.109510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baacd0-3b87-4518-8321-3d36fb8c3c97 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.109999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baacd0-3b87-4518-8321-3d36fb8c3c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.117179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 883/1771] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:06.122379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.142399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b359b3-e39c-4c4c-8577-cf2b3365e6f6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.147616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.148349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.148507 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.148851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] PATH_INFO: `/v3/projects/1a3c7d81dccb4baf956683d8229c5870/users/2ce6e7d4aaab4b48934c3b86ffb1ec93/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.154909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-02baacd0-3b87-4518-8321-3d36fb8c3c97 None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:06.156143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-02baacd0-3b87-4518-8321-3d36fb8c3c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:06.167337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3c7d81dccb4baf956683d8229c5870, user_id=2ce6e7d4aaab4b48934c3b86ffb1ec93, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.168879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 889/1772] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Sep 5 17:02:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:06.177652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baacd0-3b87-4518-8321-3d36fb8c3c97 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.179007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.195457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baacd0-3b87-4518-8321-3d36fb8c3c97 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.203206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.204139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.204580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.205071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.217428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.219489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.220037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 884/1773] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:06 2022] PUT /identity/v3/projects/1a3c7d81dccb4baf956683d8229c5870/users/2ce6e7d4aaab4b48934c3b86ffb1ec93/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:06.221224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.233825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 890/1774] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:06.349633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54275109-6542-486d-b242-6b4543004bdb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.361224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f6108-8a0d-46ec-9cad-35137bcf3f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.362062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6108-8a0d-46ec-9cad-35137bcf3f6c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.362147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6108-8a0d-46ec-9cad-35137bcf3f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.362411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6108-8a0d-46ec-9cad-35137bcf3f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.363127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f6108-8a0d-46ec-9cad-35137bcf3f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.365086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f6108-8a0d-46ec-9cad-35137bcf3f6c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.380377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 885/1775] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:06.381342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.401701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b0cfb-2a8c-4e9d-b953-9b217ee343b0 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.407380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.408894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.409606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.410049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] PATH_INFO: `/v3/projects/1a3c7d81dccb4baf956683d8229c5870/users/2ce6e7d4aaab4b48934c3b86ffb1ec93/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.412752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3c7d81dccb4baf956683d8229c5870, user_id=2ce6e7d4aaab4b48934c3b86ffb1ec93, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.416364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.433446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 891/1776] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:06 2022] PUT /identity/v3/projects/1a3c7d81dccb4baf956683d8229c5870/users/2ce6e7d4aaab4b48934c3b86ffb1ec93/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:06.446541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.447200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.447200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.484395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] MFA Rules not processed for user `2ce6e7d4aaab4b48934c3b86ffb1ec93`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:06.485631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:06.524761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:06.597067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.598047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:06.623073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 886/1777] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:06.647039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.653195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:06.706033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce7b47-a4cb-4876-a8f6-d82a91466a73 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.711605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.712368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.712746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.713086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] PATH_INFO: `/v3/projects/a0928db1d7854f1882d18f3830c57b24` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.714343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0928db1d7854f1882d18f3830c57b24)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.716411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.749841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b4489-54b4-41ed-a83f-c5e7a4a9e789 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.756033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.757283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.757601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.757877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.768692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0928db1d7854f1882d18f3830c57b24', 'request_id': 'req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.769168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.770776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.775944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0928db1d7854f1882d18f3830c57b24', 'request_id': 'req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.781813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 887/1778] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:06.782283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0928db1d7854f1882d18f3830c57b24', 'request_id': 'req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.799552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] The token cache is being invalidate because project a0928db1d7854f1882d18f3830c57b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:06.799926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.800757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.803065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 892/1779] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:06 2022] DELETE /identity/v3/projects/a0928db1d7854f1882d18f3830c57b24 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:06.807955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.821807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7874e74c-175e-40d6-b326-f6f298fc7331 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.838006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:06.840269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:06.883229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:06.890030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.890685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.890950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.891223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] PATH_INFO: `/v3/users/cc0f265709944034b8a8b3854f842074` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.892617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0f265709944034b8a8b3854f842074)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.894771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:06.911339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f9b8f-f06d-4586-93f0-e407ddee2b0b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:06.915286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:06.915923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:06.916227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:06.916511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:06.918358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:06.959869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0f265709944034b8a8b3854f842074'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.960612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0f265709944034b8a8b3854f842074'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.972084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0f265709944034b8a8b3854f842074'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.996614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:06.997437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0f265709944034b8a8b3854f842074'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.998557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0f265709944034b8a8b3854f842074'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:06.998809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:07.002210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:07.009222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 893/1780] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:07.010051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:07.011520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:07.023119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 888/1781] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:06 2022] DELETE /identity/v3/users/cc0f265709944034b8a8b3854f842074 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:07.028982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19867e4f-1849-43cf-b8f0-c9f7015e5980 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:07.093372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:07.094628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.095007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.095381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] PATH_INFO: `/v3/users/b902371c225441bf8c913979996fe262` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.097420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b902371c225441bf8c913979996fe262)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:07.099996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:07.186311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b902371c225441bf8c913979996fe262'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.200793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b902371c225441bf8c913979996fe262'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.228500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b902371c225441bf8c913979996fe262'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.229390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b902371c225441bf8c913979996fe262'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.239112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b902371c225441bf8c913979996fe262'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.248587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 894/1782] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:07 2022] DELETE /identity/v3/users/b902371c225441bf8c913979996fe262 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:07.492425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:07.494423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:07.536285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e36e4-afcc-4ab2-9af5-270c7684cb3d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:07.614334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:07.615119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.615501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.615865 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] PATH_INFO: `/v3/projects/43452046c9d54a53a8efed3ccf0fd677` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.617208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43452046c9d54a53a8efed3ccf0fd677)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:07.619489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:07.623331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:07.625526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:07.641023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b12e9-02d9-4609-a8cd-c56a1f3c8f3e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:07.646920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:07.647534 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.647850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.648227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] PATH_INFO: `/v3/projects/6268f00dde9c4e2993cbce95f65c9478` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.649942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6268f00dde9c4e2993cbce95f65c9478)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:07.652012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:07.654040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:07.695640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43452046c9d54a53a8efed3ccf0fd677', 'request_id': 'req-e4a01505-0309-42fd-acb7-201c0f8a84e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.695640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43452046c9d54a53a8efed3ccf0fd677', 'request_id': 'req-e4a01505-0309-42fd-acb7-201c0f8a84e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.698668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43452046c9d54a53a8efed3ccf0fd677', 'request_id': 'req-e4a01505-0309-42fd-acb7-201c0f8a84e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.711377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] The token cache is being invalidate because project 43452046c9d54a53a8efed3ccf0fd677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:07.711751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.712479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4a01505-0309-42fd-acb7-201c0f8a84e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.714076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 889/1783] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:07 2022] DELETE /identity/v3/projects/43452046c9d54a53a8efed3ccf0fd677 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:07.717056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6268f00dde9c4e2993cbce95f65c9478', 'request_id': 'req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.718782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da8a86-dd72-4021-a8da-aeba2259c542 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.719100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da8a86-dd72-4021-a8da-aeba2259c542 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.719345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da8a86-dd72-4021-a8da-aeba2259c542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.723388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6268f00dde9c4e2993cbce95f65c9478', 'request_id': 'req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.729880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6268f00dde9c4e2993cbce95f65c9478', 'request_id': 'req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.739375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-88da8a86-dd72-4021-a8da-aeba2259c542 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:07.740643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-88da8a86-dd72-4021-a8da-aeba2259c542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:07.744338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] The token cache is being invalidate because project 6268f00dde9c4e2993cbce95f65c9478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:07.744656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.745271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.746981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 895/1784] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:07 2022] DELETE /identity/v3/projects/6268f00dde9c4e2993cbce95f65c9478 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:07.750842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:07.752136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:07.799118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 890/1785] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:07.808318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d38ac-f240-44f6-b182-011761f9fd7a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.808694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d38ac-f240-44f6-b182-011761f9fd7a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.809038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d38ac-f240-44f6-b182-011761f9fd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.815539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4716a8b-1e79-4ca1-8da8-8aad65e0fb57 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:07.820480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:07.821188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.821460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.821729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] PATH_INFO: `/v3/users/fa9582012dde41c8878bd23f37ffe8ec` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.822855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9582012dde41c8878bd23f37ffe8ec)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:07.825285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:07.829793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-546d38ac-f240-44f6-b182-011761f9fd7a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:07.831025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-546d38ac-f240-44f6-b182-011761f9fd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:07.839160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 891/1786] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:07.843327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d38ac-f240-44f6-b182-011761f9fd7a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:07.853113 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d38ac-f240-44f6-b182-011761f9fd7a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:07.859211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:07.859817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.860090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.860403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.861051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:07.861350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:07.862955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:07.880765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9582012dde41c8878bd23f37ffe8ec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.886755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 892/1787] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:07.887955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:07.891270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9582012dde41c8878bd23f37ffe8ec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.893381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:07.901981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9993553c-fe3b-4435-93b7-0deabb28cbd6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:07.907169 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:07.907859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:07.908227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:07.908556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:07.909264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:07.909383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9582012dde41c8878bd23f37ffe8ec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.910120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9582012dde41c8878bd23f37ffe8ec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.911726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:07.919912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9582012dde41c8878bd23f37ffe8ec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:07.930748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 896/1788] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:07 2022] DELETE /identity/v3/users/fa9582012dde41c8878bd23f37ffe8ec => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:07.943463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 893/1789] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:07 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:07.948452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:07.950068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:07.962152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:08.037803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e039c4-a0fc-4a9d-a73d-4e18231052d2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.103816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.104485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.104767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.105038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.105687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.110746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.111772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.113621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:08.139726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 897/1790] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:07 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:08.139887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f890c-45bb-4f05-97c4-f266ed616c88 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.141061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.143765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.144544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.144830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.145102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] PATH_INFO: `/v3/projects/4b93ae091c9747b997baecc848881c8b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.146247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b93ae091c9747b997baecc848881c8b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.148295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.154155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41260de3-b365-45a1-9d8b-7e916cc3f9b7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.160183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.160851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.161140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.161417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.162065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.163597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.176282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 898/1791] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:08.180323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.190364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9217e2-e36d-4c27-87b5-0bef2301dcff admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.202929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b93ae091c9747b997baecc848881c8b', 'request_id': 'req-ba1d6846-65b5-4576-8e87-b1cae01b996d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:08.208805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b93ae091c9747b997baecc848881c8b', 'request_id': 'req-ba1d6846-65b5-4576-8e87-b1cae01b996d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:08.215482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b93ae091c9747b997baecc848881c8b', 'request_id': 'req-ba1d6846-65b5-4576-8e87-b1cae01b996d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:08.230287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] The token cache is being invalidate because project 4b93ae091c9747b997baecc848881c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:08.230558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:08.231194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba1d6846-65b5-4576-8e87-b1cae01b996d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:08.232740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 894/1792] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:08 2022] DELETE /identity/v3/projects/4b93ae091c9747b997baecc848881c8b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:08.238752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb3188-7f16-49d7-93ff-fc8b56a0b7f7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.239191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb3188-7f16-49d7-93ff-fc8b56a0b7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.239474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb3188-7f16-49d7-93ff-fc8b56a0b7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.253263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.253887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.254168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.254442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.255083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.256734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.260522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-06fb3188-7f16-49d7-93ff-fc8b56a0b7f7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:08.261751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-06fb3188-7f16-49d7-93ff-fc8b56a0b7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:08.267037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 899/1793] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:08.269955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 895/1794] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:08.271080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.272894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:08.277542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a17c-d7eb-4528-9ed3-89d2b827795d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.277989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a17c-d7eb-4528-9ed3-89d2b827795d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.278452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557a17c-d7eb-4528-9ed3-89d2b827795d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.286650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea28f8f-5e06-4ffe-a195-55a2867c7880 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.291263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865ae6a-fb77-47d4-9f08-c852d499774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.291940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ae6a-fb77-47d4-9f08-c852d499774a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.292286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ae6a-fb77-47d4-9f08-c852d499774a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.292546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865ae6a-fb77-47d4-9f08-c852d499774a admin admin] PATH_INFO: `/v3/projects/b2c06995fc7949a396ff8610a65f4919/users/fd7c9b81429a45d3b011488d37835b90/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.301279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8557a17c-d7eb-4528-9ed3-89d2b827795d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:08.302484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8557a17c-d7eb-4528-9ed3-89d2b827795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:08.311737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865ae6a-fb77-47d4-9f08-c852d499774a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c06995fc7949a396ff8610a65f4919, user_id=fd7c9b81429a45d3b011488d37835b90, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.313143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 896/1795] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:08.318514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557a17c-d7eb-4528-9ed3-89d2b827795d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.319940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865ae6a-fb77-47d4-9f08-c852d499774a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.331611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557a17c-d7eb-4528-9ed3-89d2b827795d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.340689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.341310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.341619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.341860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.342522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:08.342826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.344120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 900/1796] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:08 2022] PUT /identity/v3/projects/b2c06995fc7949a396ff8610a65f4919/users/fd7c9b81429a45d3b011488d37835b90/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:08.353385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.355049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae99a-d48c-4d59-a5c3-8815b307ca1c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.355326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae99a-d48c-4d59-a5c3-8815b307ca1c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.355607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ae99a-d48c-4d59-a5c3-8815b307ca1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.366996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 897/1797] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:08.371888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:08.387901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.399762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e58099-fb4c-4527-9554-c84bca02ddf2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.403892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-021ae99a-d48c-4d59-a5c3-8815b307ca1c None None] MFA Rules not processed for user `fd7c9b81429a45d3b011488d37835b90`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:08.406530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-021ae99a-d48c-4d59-a5c3-8815b307ca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:08.472622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.474113 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.474313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.474590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.475295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.477705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.520957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 901/1798] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:08.534124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae99a-d48c-4d59-a5c3-8815b307ca1c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.541685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 898/1799] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:08.543330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.551688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ae99a-d48c-4d59-a5c3-8815b307ca1c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.599504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:08.643143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8301e37-0cb1-4db6-80f1-73a38ec96c89 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.648088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.648861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.649004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.649141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.649419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.649944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.650073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.650404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.650762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.651570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.652032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.653835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.679348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 899/1800] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:08.683500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.689810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 902/1801] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:08.691987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.708051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0519-645f-49b7-8425-0de1f445d361 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.774843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988f55f-263d-4fdd-b101-e5312c785747 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.780258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.781491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.781783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.782070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.782709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.784975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.786728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.787405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.787693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.787966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.788656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.791671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.805412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 900/1802] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:08.809384 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.819574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c462cf-3644-4b9c-849f-a94c16a27dad admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.822717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 903/1803] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/users => generated 301 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:08.823625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.829039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.830568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.831002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.831478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.832136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.833849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.834715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed610200-6dee-4c7f-ab87-97105b7beb87 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.839935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.840731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.841010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.841286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.842064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.843735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.848951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 901/1804] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:08.850004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:08.854285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.860646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 904/1805] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:08.861549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.865942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97534b04-c39d-41ac-88c3-ed627a09569c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.871699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e13ade-c827-43a7-b945-6d052d010cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.872803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e13ade-c827-43a7-b945-6d052d010cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.872803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e13ade-c827-43a7-b945-6d052d010cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.873027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e13ade-c827-43a7-b945-6d052d010cec admin admin] PATH_INFO: `/v3/projects/4f9f282bfe6941c3addbb9873b45a222/users/364a99c32cd94019b68c7b702f13ea8e/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.881604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418da4b0-d8a7-437b-beb9-8e67a331f379 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:08.888722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:08.889593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.889861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.890143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] PATH_INFO: `/v3/projects/1c42833ac3fe4292bf370ac5dd013813/users/d4634f460d364ada9fc41a558aa437ea/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.907064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e13ade-c827-43a7-b945-6d052d010cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9f282bfe6941c3addbb9873b45a222, user_id=364a99c32cd94019b68c7b702f13ea8e, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.910093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e13ade-c827-43a7-b945-6d052d010cec admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.921738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c42833ac3fe4292bf370ac5dd013813, user_id=d4634f460d364ada9fc41a558aa437ea, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:08.924760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:08.933028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 902/1806] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:08 2022] PUT /identity/v3/projects/4f9f282bfe6941c3addbb9873b45a222/users/364a99c32cd94019b68c7b702f13ea8e/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:08.944983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef15f0-a5e7-4573-9ad9-53019dcbec93 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:08.945276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef15f0-a5e7-4573-9ad9-53019dcbec93 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:08.945838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 905/1807] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:08 2022] PUT /identity/v3/projects/1c42833ac3fe4292bf370ac5dd013813/users/d4634f460d364ada9fc41a558aa437ea/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:08.945998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef15f0-a5e7-4573-9ad9-53019dcbec93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:08.950482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:08.979017 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-03ef15f0-a5e7-4573-9ad9-53019dcbec93 None None] MFA Rules not processed for user `364a99c32cd94019b68c7b702f13ea8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:08.980977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-03ef15f0-a5e7-4573-9ad9-53019dcbec93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:09.035226 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51806c28-c06b-4048-942e-661b58b06860 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.041592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.042302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.042630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.042998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.043781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.045783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.059905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 906/1808] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:09.063932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.067319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 903/1809] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:09.071676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef15f0-a5e7-4573-9ad9-53019dcbec93 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.075276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f24f0-ea69-4774-9cbf-43175f9485a0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.077659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:09.080632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a62c1-076d-4081-a396-715d19cf215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.081645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a62c1-076d-4081-a396-715d19cf215c admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.081902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a62c1-076d-4081-a396-715d19cf215c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.082188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a62c1-076d-4081-a396-715d19cf215c admin admin] PATH_INFO: `/v3/projects/1c42833ac3fe4292bf370ac5dd013813/users/d4634f460d364ada9fc41a558aa437ea/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.084367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a62c1-076d-4081-a396-715d19cf215c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c42833ac3fe4292bf370ac5dd013813, user_id=d4634f460d364ada9fc41a558aa437ea, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.087904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a62c1-076d-4081-a396-715d19cf215c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.094008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef15f0-a5e7-4573-9ad9-53019dcbec93 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.104758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 907/1810] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:09 2022] PUT /identity/v3/projects/1c42833ac3fe4292bf370ac5dd013813/users/d4634f460d364ada9fc41a558aa437ea/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:09.115127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.115560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.115893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.150726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] MFA Rules not processed for user `d4634f460d364ada9fc41a558aa437ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:09.151963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:09.162490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.163121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.163405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.163687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.164504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.166471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.198037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 904/1811] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:09 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:09.198982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.224894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 908/1812] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:09.236937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.238391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:09.263156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751ab8c-bd4a-4523-ae7c-16cec669a0ad admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.268763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.269381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.269658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.269928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.270566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.272543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.286425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:09.296905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 905/1813] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:09 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:09.298490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43f44b-762c-491d-9547-15d00866ef32 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.300772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.310540 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ee73-b116-44cc-a1fa-4a87b67df03b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.315393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.316008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.316320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.316586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.317215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.318804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.329474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 906/1814] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:09.333280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.344536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab712c7-97b4-48a5-9fc6-ec8171bed579 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.349200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.349851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.350131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.350401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.351031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.351267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.352099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.352882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.353161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.353253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.363245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 907/1815] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:09.367578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.377859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937b372-9a6a-4e18-a3ae-09a8904b9d7b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.383193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f1a2e-76ec-47ee-9174-b17ebe005c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.384097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f1a2e-76ec-47ee-9174-b17ebe005c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.384430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f1a2e-76ec-47ee-9174-b17ebe005c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.384751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f1a2e-76ec-47ee-9174-b17ebe005c14 admin admin] PATH_INFO: `/v3/projects/681876fba5154f07bd64f7efb909c184/users/be5c29d5b4da45dab6e5c3b11b87fa10/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.401239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f1a2e-76ec-47ee-9174-b17ebe005c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681876fba5154f07bd64f7efb909c184, user_id=be5c29d5b4da45dab6e5c3b11b87fa10, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.403935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f1a2e-76ec-47ee-9174-b17ebe005c14 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.418008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.418955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 908/1816] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:09 2022] PUT /identity/v3/projects/681876fba5154f07bd64f7efb909c184/users/be5c29d5b4da45dab6e5c3b11b87fa10/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:09.419605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.429019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0156326-f21b-40a4-8d37-f79fcf0c68a0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.429308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0156326-f21b-40a4-8d37-f79fcf0c68a0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.429584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0156326-f21b-40a4-8d37-f79fcf0c68a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.429858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 909/1817] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:09.458575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c0156326-f21b-40a4-8d37-f79fcf0c68a0 None None] MFA Rules not processed for user `be5c29d5b4da45dab6e5c3b11b87fa10`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:09.459860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c0156326-f21b-40a4-8d37-f79fcf0c68a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:09.484091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:09.539827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 909/1818] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:09.541359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.562905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2c3dd-d4a2-4e92-8fec-746f7a91a907 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.654504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.655368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.655751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.656648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.657620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.659870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.692636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 910/1819] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:09 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:09.697528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0156326-f21b-40a4-8d37-f79fcf0c68a0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.712361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0156326-f21b-40a4-8d37-f79fcf0c68a0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.734391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.783021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:09.811816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.812641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.812987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.813343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.814042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.814274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa1000-0fb9-477e-9361-baecc82774ac admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.816075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.818823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.819459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.819737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.820010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] PATH_INFO: `/v3/users/fd7c9b81429a45d3b011488d37835b90` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.821562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7c9b81429a45d3b011488d37835b90)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.823670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.849996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 910/1820] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:09 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:09.854639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.864728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1f0cc-db94-49d7-8b36-eb79183ead26 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.870016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.870623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.870901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.871172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.871815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.873476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.875611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7c9b81429a45d3b011488d37835b90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:09.884280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7c9b81429a45d3b011488d37835b90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:09.888928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 911/1821] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:09.889968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.897015 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7c9b81429a45d3b011488d37835b90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:09.897706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7c9b81429a45d3b011488d37835b90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:09.900129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1924fb-36ea-4eb3-b57d-57e67b555c6a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:09.904967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:09.905095 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5d7a969d-ba0e-4122-bc74-3c5dd822a1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7c9b81429a45d3b011488d37835b90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:09.905610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.905913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.906309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] PATH_INFO: `/v3/projects/9691026dedfd4e30979126c4de3ae022/users/5b2ed2d5706b469387521bf06b1db645/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.911296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 911/1822] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:09 2022] DELETE /identity/v3/users/fd7c9b81429a45d3b011488d37835b90 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:09.916313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:09.916592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:09.916861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:09.921264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9691026dedfd4e30979126c4de3ae022, user_id=5b2ed2d5706b469387521bf06b1db645, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:09.924179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:09.937766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:09.938844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:09.940805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 912/1823] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:09 2022] PUT /identity/v3/projects/9691026dedfd4e30979126c4de3ae022/users/5b2ed2d5706b469387521bf06b1db645/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:09.944579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:09.946651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.025213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef8162-b640-4a0d-9059-be9c54a5f9f3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.026587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 912/1824] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:10.031840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.032499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.032774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.033045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.033686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.035317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.036967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:10.038356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.047106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 913/1825] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:10.051225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:10.062291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f4a7-cb0c-4fae-86c6-dc2f6099efaa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.068775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c56ad4-cae0-46e0-a91b-84735cccdad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.069446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56ad4-cae0-46e0-a91b-84735cccdad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.070088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56ad4-cae0-46e0-a91b-84735cccdad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.070946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c56ad4-cae0-46e0-a91b-84735cccdad4 admin admin] PATH_INFO: `/v3/projects/9691026dedfd4e30979126c4de3ae022/users/5b2ed2d5706b469387521bf06b1db645/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.072882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c56ad4-cae0-46e0-a91b-84735cccdad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9691026dedfd4e30979126c4de3ae022, user_id=5b2ed2d5706b469387521bf06b1db645, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.076023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c56ad4-cae0-46e0-a91b-84735cccdad4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.089210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:10.100317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 914/1826] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:10 2022] PUT /identity/v3/projects/9691026dedfd4e30979126c4de3ae022/users/5b2ed2d5706b469387521bf06b1db645/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:10.107773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.108163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.108512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.114206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26605935-dfd0-42f9-8386-13b3ba83941b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.122821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:10.145403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] MFA Rules not processed for user `5b2ed2d5706b469387521bf06b1db645`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:10.146828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.171730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.172394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.172678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.172956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.232726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 915/1827] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:10.240322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:10.241916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.248133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.249697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.262880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 913/1828] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:10.309284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b1d95-81fb-446e-a6f2-c1f226f35448 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.313301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.313931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.314350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.314517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.324961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.326435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.338629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 916/1829] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:10.395982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:10.406841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f076-d7ce-4e27-9b07-a641ebf46d39 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.413038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.414330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.415659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.416110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.419845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.505032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.506880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.518563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 914/1830] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:10.549595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:10.551016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.576474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227b2c1-ece1-4388-b3e4-0ba45bc9376f service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.582597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.583289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.583514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.583781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] PATH_INFO: `/v3/projects/b2c06995fc7949a396ff8610a65f4919` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.585680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c06995fc7949a396ff8610a65f4919)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.589472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.649593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:10.650960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.651450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c06995fc7949a396ff8610a65f4919', 'request_id': 'req-0b2c2ec5-1cdd-4971-b341-423da7490750'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.657502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c06995fc7949a396ff8610a65f4919', 'request_id': 'req-0b2c2ec5-1cdd-4971-b341-423da7490750'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.661813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef049fb-10e8-44cf-b881-dd993fd75bbc service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.664104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c06995fc7949a396ff8610a65f4919', 'request_id': 'req-0b2c2ec5-1cdd-4971-b341-423da7490750'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.681066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] The token cache is being invalidate because project b2c06995fc7949a396ff8610a65f4919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:10.681380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.682004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.683739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 917/1831] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:10 2022] DELETE /identity/v3/projects/b2c06995fc7949a396ff8610a65f4919 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:10.687622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:10.689110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.700681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:10.725085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.725716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.726153 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.726436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.729382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:10.744891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:10.773187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c2ec5-1cdd-4971-b341-423da7490750 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:10.780882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:10.781431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:10.781709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:10.781979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] PATH_INFO: `/v3/users/d4634f460d364ada9fc41a558aa437ea` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:10.783763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4634f460d364ada9fc41a558aa437ea)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.786555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.821700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:10.823362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:10.843826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4634f460d364ada9fc41a558aa437ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.844655 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4634f460d364ada9fc41a558aa437ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.854844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 915/1832] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:10.868486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4634f460d364ada9fc41a558aa437ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.902339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4634f460d364ada9fc41a558aa437ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.904294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4634f460d364ada9fc41a558aa437ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:10.923643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 918/1833] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:10 2022] DELETE /identity/v3/users/d4634f460d364ada9fc41a558aa437ea => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:11.145165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.146646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:11.172547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed840c-ad12-46dd-81d6-5fc906359ba3 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.272707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.273693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.274692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.275030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] PATH_INFO: `/v3/users/a4ee9da54301496e9d7f383036e1c22c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.276414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ee9da54301496e9d7f383036e1c22c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.279124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.341475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ee9da54301496e9d7f383036e1c22c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.359964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ee9da54301496e9d7f383036e1c22c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.363716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.365116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:11.379180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba082c4a-ad59-49bd-9732-e8680797eed6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.387375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.387943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.388449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.388553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] PATH_INFO: `/v3/projects/1c42833ac3fe4292bf370ac5dd013813` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.390298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ee9da54301496e9d7f383036e1c22c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.391108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c42833ac3fe4292bf370ac5dd013813)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.391262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ee9da54301496e9d7f383036e1c22c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.393253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.400279 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96b83482-6700-426d-84bc-726eac130a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ee9da54301496e9d7f383036e1c22c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.408077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 916/1834] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:11 2022] DELETE /identity/v3/users/a4ee9da54301496e9d7f383036e1c22c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:11.412657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.413041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.413425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.452566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c42833ac3fe4292bf370ac5dd013813', 'request_id': 'req-687c8685-024a-47d5-9512-8d89972b8590'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.454781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:11.456738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:11.462358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c42833ac3fe4292bf370ac5dd013813', 'request_id': 'req-687c8685-024a-47d5-9512-8d89972b8590'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.469474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c42833ac3fe4292bf370ac5dd013813', 'request_id': 'req-687c8685-024a-47d5-9512-8d89972b8590'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.484579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] The token cache is being invalidate because project 1c42833ac3fe4292bf370ac5dd013813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:11.484957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.485671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-687c8685-024a-47d5-9512-8d89972b8590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.487380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 919/1835] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:11 2022] DELETE /identity/v3/projects/1c42833ac3fe4292bf370ac5dd013813 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:11.488263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:11.492117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de7f0-4aea-4786-8b87-d0a2515d9f8b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.492695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de7f0-4aea-4786-8b87-d0a2515d9f8b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.493084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989de7f0-4aea-4786-8b87-d0a2515d9f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.517830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-989de7f0-4aea-4786-8b87-d0a2515d9f8b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:11.519042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-989de7f0-4aea-4786-8b87-d0a2515d9f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:11.524844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:11.527019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 917/1836] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:11.537626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.539293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:11.583432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 920/1837] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:11.592294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df334e6d-ca5e-42d2-b4fa-53110bc2193b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.592576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df334e6d-ca5e-42d2-b4fa-53110bc2193b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.592844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df334e6d-ca5e-42d2-b4fa-53110bc2193b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.599691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff604178-d511-407d-a48c-ab0f8d8411c0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.603825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.604492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.604862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.605191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.612731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-df334e6d-ca5e-42d2-b4fa-53110bc2193b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:11.613915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-df334e6d-ca5e-42d2-b4fa-53110bc2193b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:11.614736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.616244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.622965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 921/1838] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:11.625956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 918/1839] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:11.629127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df334e6d-ca5e-42d2-b4fa-53110bc2193b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.638347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df334e6d-ca5e-42d2-b4fa-53110bc2193b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.643239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.643874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.644177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.644445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.645095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:11.645389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.647156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.660191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 922/1840] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:11.663585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.674185 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e7dc3-e907-4290-9991-837ddb020512 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.679083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.679691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.680163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.680446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.681293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.683299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.712799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 919/1841] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:11 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:11.717104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.728014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1f754-d6e9-44da-9c8f-444fcba5b623 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.795332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.795964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.796279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.796558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.797216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.799151 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.821958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 923/1842] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:11 2022] POST /identity/v3/users => generated 305 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:11.826191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.837566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ee577-359a-42fe-8206-5255488a7b5b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.842422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.843133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.843515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.843964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.844653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.846329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.857559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 920/1843] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:11.861755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.872478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc222d7-bb17-40c0-9f8d-d64c1f9e9104 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.877171 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.877777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.878054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.878321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.878962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.880690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.893782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 924/1844] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:11.899968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.900950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:11.903035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:11.909359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2bc5d-c69e-4a4c-bd45-a20662021f87 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.915049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea634b-e79c-4f86-a359-78587355f60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.916016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea634b-e79c-4f86-a359-78587355f60e admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.916384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea634b-e79c-4f86-a359-78587355f60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.916723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea634b-e79c-4f86-a359-78587355f60e admin admin] PATH_INFO: `/v3/projects/79e2a7b2d80c44559d5a6efcf7309974/users/ad3a8a893c9742fab21bed017f981e25/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.918525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f7559-cc92-4c26-b42d-90d26b30dc46 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:11.923447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:11.924081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.924387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.924658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] PATH_INFO: `/v3/projects/690a4d84f91144ad8bd5a9f8f51b8449` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.925776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=690a4d84f91144ad8bd5a9f8f51b8449)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.928363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.934078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea634b-e79c-4f86-a359-78587355f60e admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e2a7b2d80c44559d5a6efcf7309974, user_id=ad3a8a893c9742fab21bed017f981e25, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:11.936996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea634b-e79c-4f86-a359-78587355f60e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:11.953211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 921/1845] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:11 2022] PUT /identity/v3/projects/79e2a7b2d80c44559d5a6efcf7309974/users/ad3a8a893c9742fab21bed017f981e25/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:11.964922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:11.965212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:11.965603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:11.984566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690a4d84f91144ad8bd5a9f8f51b8449', 'request_id': 'req-c595a865-7121-458b-aaf0-ba677e57a794'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.992955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690a4d84f91144ad8bd5a9f8f51b8449', 'request_id': 'req-c595a865-7121-458b-aaf0-ba677e57a794'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:11.996181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] MFA Rules not processed for user `ad3a8a893c9742fab21bed017f981e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:11.997411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:12.000078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690a4d84f91144ad8bd5a9f8f51b8449', 'request_id': 'req-c595a865-7121-458b-aaf0-ba677e57a794'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.005308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:12.021128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] The token cache is being invalidate because project 690a4d84f91144ad8bd5a9f8f51b8449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:12.021876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.022977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.028352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 925/1846] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:11 2022] DELETE /identity/v3/projects/690a4d84f91144ad8bd5a9f8f51b8449 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:12.029538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.030730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:12.063736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 922/1847] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:12.068505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.069868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:12.077051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:12.102141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595a865-7121-458b-aaf0-ba677e57a794 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.107900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.108928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.109291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.109643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] PATH_INFO: `/v3/users/2ce6e7d4aaab4b48934c3b86ffb1ec93` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.112479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce6e7d4aaab4b48934c3b86ffb1ec93)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.114566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.144675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d1285-7e7d-4c9f-bfab-b1b3a9b1ae2c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.152980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.154364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.154896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.155534 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.156459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.160878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.179706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce6e7d4aaab4b48934c3b86ffb1ec93'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.180479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce6e7d4aaab4b48934c3b86ffb1ec93'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.192592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce6e7d4aaab4b48934c3b86ffb1ec93'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.207284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 923/1848] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:12 2022] POST /identity/v3/projects => generated 409 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:12.211738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.212651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce6e7d4aaab4b48934c3b86ffb1ec93'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.213703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce6e7d4aaab4b48934c3b86ffb1ec93'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.216302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:12.239291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 926/1849] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:12 2022] DELETE /identity/v3/users/2ce6e7d4aaab4b48934c3b86ffb1ec93 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:12.298329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaf817-0a94-4d64-8fd6-9e11439fc29a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.368045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.368812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.369275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.369606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.370270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.373075 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.402061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 924/1850] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:12 2022] POST /identity/v3/users => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:12.405983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.418027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cf826-a2d9-4850-8ec4-11d4c490109d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.422982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.423597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.423891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.424186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.424847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.426559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.441044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 927/1851] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:12.445457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.456606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321ecad-a7a1-449d-9293-4cd935fa1718 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.461254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.461929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.462216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.462494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] PATH_INFO: `/v3/projects/3dc4653d5c264edd8a7a799332618046/users/a16dd68db91844f1a2a0321fcb95d1b1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.480466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc4653d5c264edd8a7a799332618046, user_id=a16dd68db91844f1a2a0321fcb95d1b1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.483680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.503479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 925/1852] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:12 2022] PUT /identity/v3/projects/3dc4653d5c264edd8a7a799332618046/users/a16dd68db91844f1a2a0321fcb95d1b1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:12.505029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.515335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a210fd5-576f-49c8-bd2b-63c956a06f6b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.573221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deea1c-e92b-4909-851d-cc0c7b312ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.573858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deea1c-e92b-4909-851d-cc0c7b312ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.574140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deea1c-e92b-4909-851d-cc0c7b312ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.574444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deea1c-e92b-4909-851d-cc0c7b312ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.575100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8deea1c-e92b-4909-851d-cc0c7b312ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.576777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8deea1c-e92b-4909-851d-cc0c7b312ac9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.579351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:12.590704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 928/1853] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:12.593383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.599021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:12.605903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92338b1f-abcc-405f-82c0-31c0a42579f8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.611688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.612424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.612712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.613020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] PATH_INFO: `/v3/projects/3dc4653d5c264edd8a7a799332618046/users/a16dd68db91844f1a2a0321fcb95d1b1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.615334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc4653d5c264edd8a7a799332618046, user_id=a16dd68db91844f1a2a0321fcb95d1b1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.618319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.629939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 926/1854] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:12 2022] PUT /identity/v3/projects/3dc4653d5c264edd8a7a799332618046/users/a16dd68db91844f1a2a0321fcb95d1b1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:12.640998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.641284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.641557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.672114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] MFA Rules not processed for user `a16dd68db91844f1a2a0321fcb95d1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:12.673393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:12.748175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 929/1855] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:12 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:12.759525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.761486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:12.825451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:12.827081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:12.828404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12fa56-9965-4a07-a376-ca30f377a60f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.832786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7561256-630f-4d9f-a1d2-7922325c4b1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.833799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7561256-630f-4d9f-a1d2-7922325c4b1b service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.834071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7561256-630f-4d9f-a1d2-7922325c4b1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.834449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7561256-630f-4d9f-a1d2-7922325c4b1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.847186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7561256-630f-4d9f-a1d2-7922325c4b1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.849538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7561256-630f-4d9f-a1d2-7922325c4b1b service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.861808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 927/1856] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:12 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:12.902067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b4a34-cdee-43d0-a444-0e9b6f6662e1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:12.908240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:12.908240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.908240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.908503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] PATH_INFO: `/v3/projects/1a3c7d81dccb4baf956683d8229c5870` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:12.909580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a3c7d81dccb4baf956683d8229c5870)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:12.911549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:12.960173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a3c7d81dccb4baf956683d8229c5870', 'request_id': 'req-0a4a36e0-edeb-4647-a752-aff35326fbc2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.966065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a3c7d81dccb4baf956683d8229c5870', 'request_id': 'req-0a4a36e0-edeb-4647-a752-aff35326fbc2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.972638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a3c7d81dccb4baf956683d8229c5870', 'request_id': 'req-0a4a36e0-edeb-4647-a752-aff35326fbc2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.988659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] The token cache is being invalidate because project 1a3c7d81dccb4baf956683d8229c5870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:12.988971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.989604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:12.991193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 930/1857] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:12 2022] DELETE /identity/v3/projects/1a3c7d81dccb4baf956683d8229c5870 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:12.997399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab121c9-47e1-4cc4-abc1-f58d9e35fd8f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:12.997702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab121c9-47e1-4cc4-abc1-f58d9e35fd8f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:12.997980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab121c9-47e1-4cc4-abc1-f58d9e35fd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.019007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-eab121c9-47e1-4cc4-abc1-f58d9e35fd8f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:13.020262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eab121c9-47e1-4cc4-abc1-f58d9e35fd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:13.049751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.051549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:13.070102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:13.093449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 928/1858] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:13.101422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4014b8e-dd72-4c07-aadf-c6d72d551baa None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.102009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4014b8e-dd72-4c07-aadf-c6d72d551baa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.102551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4014b8e-dd72-4c07-aadf-c6d72d551baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.121466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a36e0-edeb-4647-a752-aff35326fbc2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.124361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b4014b8e-dd72-4c07-aadf-c6d72d551baa None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:13.125005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.125566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b4014b8e-dd72-4c07-aadf-c6d72d551baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:13.125678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.125892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.126164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.127876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:13.133714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 929/1859] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:13.138024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4014b8e-dd72-4c07-aadf-c6d72d551baa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.147034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4014b8e-dd72-4c07-aadf-c6d72d551baa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.151855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.152489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.152776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.153055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.153699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:13.154010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.155660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.167293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 930/1860] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:02:13.171689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.181397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44daf1-92cc-49d3-9310-b6146d7d67be admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.186054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.186665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.186948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.187217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.187836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.189780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.189914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.192018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.202982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 931/1861] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:13.230805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 931/1862] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:13 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:13.231947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.243702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b4810-6d8d-47f8-9775-8833fb3894f7 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.304336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.305114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.305296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.305593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.306158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.308703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.334394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 932/1863] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:13 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:13.340014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.354404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c514b5-28da-4004-b045-c25ff02f300c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.359902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.360559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.360814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.361473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.362243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.363999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.381333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 932/1864] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:13.383909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.394779 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0d6ba-7d9c-43a6-b996-06fe01431c08 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.400503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652bb26-357c-4294-a6ad-bd29890d1bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.401193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652bb26-357c-4294-a6ad-bd29890d1bde admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.401530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652bb26-357c-4294-a6ad-bd29890d1bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.401859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652bb26-357c-4294-a6ad-bd29890d1bde admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.402708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652bb26-357c-4294-a6ad-bd29890d1bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.404677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652bb26-357c-4294-a6ad-bd29890d1bde admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.415694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 933/1865] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:13.420118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.435195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b67e2-180e-4d63-bcd0-d30fcf123758 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.440955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.441850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.442138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.442420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] PATH_INFO: `/v3/projects/4afec509ee454beba68b0a8801edae64/users/45e8235b38184240ab7d7397d5603c15/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.464606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4afec509ee454beba68b0a8801edae64, user_id=45e8235b38184240ab7d7397d5603c15, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.470437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.493341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 933/1866] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:13 2022] PUT /identity/v3/projects/4afec509ee454beba68b0a8801edae64/users/45e8235b38184240ab7d7397d5603c15/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:13.509855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eb54f-f07b-4b24-b97a-dc33549cf1c8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.510261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eb54f-f07b-4b24-b97a-dc33549cf1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.510628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eb54f-f07b-4b24-b97a-dc33549cf1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.565699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2d9eb54f-f07b-4b24-b97a-dc33549cf1c8 None None] MFA Rules not processed for user `45e8235b38184240ab7d7397d5603c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:13.567166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d9eb54f-f07b-4b24-b97a-dc33549cf1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:13.620180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:13.660344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 934/1867] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:13.672287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:13.674103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:13.693581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:13.763029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5008d-41b3-4b65-8a30-0d4b48e8f09d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:13.766850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33b416-0203-4150-9cb6-1f2d1a8ecc66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05a016553950460dbb736ced2cae4f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:13.767464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33b416-0203-4150-9cb6-1f2d1a8ecc66 service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:13.767741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33b416-0203-4150-9cb6-1f2d1a8ecc66 service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:13.768011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33b416-0203-4150-9cb6-1f2d1a8ecc66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:13.776525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33b416-0203-4150-9cb6-1f2d1a8ecc66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:13.777970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33b416-0203-4150-9cb6-1f2d1a8ecc66 service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:13.788682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 934/1868] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:19.467888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc0b7d-c263-4d3e-bcd2-9686e9a63f1f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:19.467888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc0b7d-c263-4d3e-bcd2-9686e9a63f1f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:19.467888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc0b7d-c263-4d3e-bcd2-9686e9a63f1f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:19.473505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 935/1869] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:02:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:02:19.481545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:19.482498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:19.482894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:19.509957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:19.511749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:19.608595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 935/1870] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:02:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:19.622854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc0b7d-c263-4d3e-bcd2-9686e9a63f1f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:19.625002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a8fc0b7d-c263-4d3e-bcd2-9686e9a63f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:19.643654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc0b7d-c263-4d3e-bcd2-9686e9a63f1f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:19.649945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2b7b6-f98b-4753-8bca-b4404cb7170c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:19.650572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2b7b6-f98b-4753-8bca-b4404cb7170c service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:19.650853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2b7b6-f98b-4753-8bca-b4404cb7170c service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:19.651203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2b7b6-f98b-4753-8bca-b4404cb7170c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:19.661445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2b7b6-f98b-4753-8bca-b4404cb7170c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:19.663112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2b7b6-f98b-4753-8bca-b4404cb7170c service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:19.674770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 936/1871] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:02:26.596731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:26.598360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:26.665950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52261df3-feca-4bf0-8a02-765535a24bda None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:26.672344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:26.673196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:26.673677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:26.674332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] PATH_INFO: `/v3/users/364a99c32cd94019b68c7b702f13ea8e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:26.675785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=364a99c32cd94019b68c7b702f13ea8e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:26.679058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:26.722371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:26.738272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364a99c32cd94019b68c7b702f13ea8e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:26.747165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364a99c32cd94019b68c7b702f13ea8e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:26.759397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364a99c32cd94019b68c7b702f13ea8e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:26.760053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364a99c32cd94019b68c7b702f13ea8e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:26.767693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364a99c32cd94019b68c7b702f13ea8e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:26.774261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 936/1872] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:26 2022] DELETE /identity/v3/users/364a99c32cd94019b68c7b702f13ea8e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:26.778909 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:26.779377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:26.779464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:26.803842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:26.805076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:26.868858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 937/1873] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:26.878628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:26.881805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:26.946616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e76b2b-f0ac-491c-849b-b1d2ad54aecb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:26.951494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:26.952121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:26.952440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:26.952752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:26.974445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:26.976132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:26.987891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 937/1874] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:27.335698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:27.337343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:27.350591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141863d-0ec1-475e-9a18-f2e93de28cca None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:27.358776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:27.358776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:27.358776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:27.358776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] PATH_INFO: `/v3/projects/4f9f282bfe6941c3addbb9873b45a222` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:27.359502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9f282bfe6941c3addbb9873b45a222)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:27.360023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:27.417546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9f282bfe6941c3addbb9873b45a222', 'request_id': 'req-c1d34c9d-58bc-4ed4-b53c-675da322be70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.423551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9f282bfe6941c3addbb9873b45a222', 'request_id': 'req-c1d34c9d-58bc-4ed4-b53c-675da322be70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.429693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9f282bfe6941c3addbb9873b45a222', 'request_id': 'req-c1d34c9d-58bc-4ed4-b53c-675da322be70'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.444887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] The token cache is being invalidate because project 4f9f282bfe6941c3addbb9873b45a222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:27.445178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.445812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.447317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 938/1875] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:27 2022] DELETE /identity/v3/projects/4f9f282bfe6941c3addbb9873b45a222 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:27.451494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:27.462780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376ae9f3-586b-4c69-8abc-58c0cb0081f6 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:27.520280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:27.520931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:27.521212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:27.521484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] PATH_INFO: `/v3/users/be5c29d5b4da45dab6e5c3b11b87fa10` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:27.522617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5c29d5b4da45dab6e5c3b11b87fa10)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:27.524709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:27.574302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5c29d5b4da45dab6e5c3b11b87fa10'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.583889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5c29d5b4da45dab6e5c3b11b87fa10'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.585289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:27.604491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5c29d5b4da45dab6e5c3b11b87fa10'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.605991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5c29d5b4da45dab6e5c3b11b87fa10'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.613192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5c29d5b4da45dab6e5c3b11b87fa10'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:27.619567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 938/1876] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:27 2022] DELETE /identity/v3/users/be5c29d5b4da45dab6e5c3b11b87fa10 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:27.909547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:27.911143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:27.925271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d34c9d-58bc-4ed4-b53c-675da322be70 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:27.971040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:27.988567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:27.989348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:27.989630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:27.989935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] PATH_INFO: `/v3/projects/681876fba5154f07bd64f7efb909c184` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:27.991152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681876fba5154f07bd64f7efb909c184)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:27.993448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:28.043626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681876fba5154f07bd64f7efb909c184', 'request_id': 'req-db7c87f0-e205-4390-9dbe-37774e461de7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.049793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681876fba5154f07bd64f7efb909c184', 'request_id': 'req-db7c87f0-e205-4390-9dbe-37774e461de7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.056378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681876fba5154f07bd64f7efb909c184', 'request_id': 'req-db7c87f0-e205-4390-9dbe-37774e461de7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.071253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] The token cache is being invalidate because project 681876fba5154f07bd64f7efb909c184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:28.071545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.072196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.073827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 939/1877] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:27 2022] DELETE /identity/v3/projects/681876fba5154f07bd64f7efb909c184 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:28.078039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:28.079669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:28.093300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f8374-4c03-4a9a-9615-ad285a8fb1ba admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:28.153501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:28.154159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:28.154436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:28.154711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] PATH_INFO: `/v3/users/5b2ed2d5706b469387521bf06b1db645` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:28.155836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2ed2d5706b469387521bf06b1db645)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:28.157932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:28.212342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2ed2d5706b469387521bf06b1db645'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.222697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2ed2d5706b469387521bf06b1db645'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.238639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2ed2d5706b469387521bf06b1db645'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.239376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2ed2d5706b469387521bf06b1db645'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.246782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e822992-606c-4a27-b446-c4cc1e1d3865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2ed2d5706b469387521bf06b1db645'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.256190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 939/1878] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:28 2022] DELETE /identity/v3/users/5b2ed2d5706b469387521bf06b1db645 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:28.544185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:28.546116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:28.559878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c87f0-e205-4390-9dbe-37774e461de7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:28.621596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:28.622270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:28.622548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:28.622815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] PATH_INFO: `/v3/projects/9691026dedfd4e30979126c4de3ae022` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:28.624087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9691026dedfd4e30979126c4de3ae022)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:28.626557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:28.680662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9691026dedfd4e30979126c4de3ae022', 'request_id': 'req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.689627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9691026dedfd4e30979126c4de3ae022', 'request_id': 'req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.696184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9691026dedfd4e30979126c4de3ae022', 'request_id': 'req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.713330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] The token cache is being invalidate because project 9691026dedfd4e30979126c4de3ae022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:28.713661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.714935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-79ef2669-4fbc-47bb-b5f2-c7c3a21f1d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:28.720203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 940/1879] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:28 2022] DELETE /identity/v3/projects/9691026dedfd4e30979126c4de3ae022 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:28.724418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766805f-f40f-4d52-900c-712befd63301 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:28.724945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766805f-f40f-4d52-900c-712befd63301 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:28.724945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766805f-f40f-4d52-900c-712befd63301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:28.750754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5766805f-f40f-4d52-900c-712befd63301 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:28.752092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5766805f-f40f-4d52-900c-712befd63301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:28.808271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:28.827480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 940/1880] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:28.835558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc8be-772d-4b0d-93ee-eb681db334b9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:28.835836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc8be-772d-4b0d-93ee-eb681db334b9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:28.836113 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfc8be-772d-4b0d-93ee-eb681db334b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:28.851646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:28.857771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f3cfc8be-772d-4b0d-93ee-eb681db334b9 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:28.859130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3cfc8be-772d-4b0d-93ee-eb681db334b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:28.872717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 941/1881] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:28.877328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766805f-f40f-4d52-900c-712befd63301 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:28.887573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766805f-f40f-4d52-900c-712befd63301 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:28.893812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:28.894542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:28.894832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:28.895083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:28.895748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:28.896053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:28.897727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:28.911063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 941/1882] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:02:28.915836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfc8be-772d-4b0d-93ee-eb681db334b9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:28.929659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfc8be-772d-4b0d-93ee-eb681db334b9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:28.934369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:28.935001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:28.935285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:28.935533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:28.936180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:28.938138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:28.972242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 942/1883] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:28 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:28.976734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:28.987715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365fe581-ffff-4b37-802b-8469e2cd5bbd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:29.046863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:29.047778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:29.048059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:29.048358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:29.048991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:29.050942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:29.079106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 942/1884] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:28 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:29.080287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:29.092431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb3cf-8bae-4732-a04d-78539503ac89 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:29.097372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:29.097951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:29.098235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:29.098526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:29.099167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:29.100770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:29.112277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 943/1885] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:29.116486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:29.127415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560576-75b7-480a-a67e-d7302f4a8e96 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:29.132603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fa26b-c6c1-40eb-9279-bad1697da407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:29.133219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fa26b-c6c1-40eb-9279-bad1697da407 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:29.133528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fa26b-c6c1-40eb-9279-bad1697da407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:29.133805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fa26b-c6c1-40eb-9279-bad1697da407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:29.134444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fa26b-c6c1-40eb-9279-bad1697da407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:29.136067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fa26b-c6c1-40eb-9279-bad1697da407 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:29.147398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 943/1886] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:29.151618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:29.161808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c8264-de23-493b-b9aa-35ed1b1a6df8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:29.166485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:29.167153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:29.167427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:29.167697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] PATH_INFO: `/v3/projects/1ff96d76ce9e432fa63aa8a2feb80ec9/users/c73df39bdc524ea2a914cc8c732e6839/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:29.182778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ff96d76ce9e432fa63aa8a2feb80ec9, user_id=c73df39bdc524ea2a914cc8c732e6839, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:29.185477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:29.205186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 944/1887] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:29 2022] PUT /identity/v3/projects/1ff96d76ce9e432fa63aa8a2feb80ec9/users/c73df39bdc524ea2a914cc8c732e6839/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:29.223076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:29.223324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:29.223619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:29.253821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] MFA Rules not processed for user `c73df39bdc524ea2a914cc8c732e6839`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:29.254985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:29.328445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 944/1888] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:29.341046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:29.343179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:29.416749 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108e2e1-cc22-4b08-a0a3-191c226d3be7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:29.421261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f435a-c525-4b7e-a4f9-3756e252e7ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:29.421956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f435a-c525-4b7e-a4f9-3756e252e7ec service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:29.422504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f435a-c525-4b7e-a4f9-3756e252e7ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:29.423190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f435a-c525-4b7e-a4f9-3756e252e7ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:29.437855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f435a-c525-4b7e-a4f9-3756e252e7ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:29.437855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f435a-c525-4b7e-a4f9-3756e252e7ec service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:29.447038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 945/1889] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:39.325127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:39.443064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:52.696087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:52.697844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:52.768559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5593d0-75fb-4b7e-9c63-5c1ccfc454a4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:52.773213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:52.773866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:52.774139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:52.774410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] PATH_INFO: `/v3/users/ad3a8a893c9742fab21bed017f981e25` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:52.775517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3a8a893c9742fab21bed017f981e25)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:52.777474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:52.827824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3a8a893c9742fab21bed017f981e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:52.836941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3a8a893c9742fab21bed017f981e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:52.853967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3a8a893c9742fab21bed017f981e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:52.854721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3a8a893c9742fab21bed017f981e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:52.862332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3a8a893c9742fab21bed017f981e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:52.868323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 945/1890] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:52 2022] DELETE /identity/v3/users/ad3a8a893c9742fab21bed017f981e25 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:52.872939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:52.873446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:52.873446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:52.893577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:52.894765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:52.955501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 946/1891] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:52.965672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:52.967588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:53.030842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddd7af-5e64-4a8e-8a44-e50fd07202a4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:53.036788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:53.037620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:53.037897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:53.038166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:53.048261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:53.049737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:53.060418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 946/1892] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:53.353617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:53.355258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:53.368892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ce086-00d3-4bdb-9c3e-fdf0f26a6787 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:53.374054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:53.374721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:53.375000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:53.375269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] PATH_INFO: `/v3/projects/79e2a7b2d80c44559d5a6efcf7309974` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:53.376463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e2a7b2d80c44559d5a6efcf7309974)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:53.378380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:53.431088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e2a7b2d80c44559d5a6efcf7309974', 'request_id': 'req-bf9e6de0-1a63-48af-b47e-53c26face28e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.436634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e2a7b2d80c44559d5a6efcf7309974', 'request_id': 'req-bf9e6de0-1a63-48af-b47e-53c26face28e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.443365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e2a7b2d80c44559d5a6efcf7309974', 'request_id': 'req-bf9e6de0-1a63-48af-b47e-53c26face28e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.456964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] The token cache is being invalidate because project 79e2a7b2d80c44559d5a6efcf7309974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:53.457295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.457932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.459688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 947/1893] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:53 2022] DELETE /identity/v3/projects/79e2a7b2d80c44559d5a6efcf7309974 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:53.464015 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:53.474425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7ec39-64ec-45fc-9f04-19924343494a service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:53.494663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:53.538207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:53.538875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:53.539149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:53.539416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] PATH_INFO: `/v3/users/a16dd68db91844f1a2a0321fcb95d1b1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:53.540656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16dd68db91844f1a2a0321fcb95d1b1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:53.542583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:53.591753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16dd68db91844f1a2a0321fcb95d1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.600527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16dd68db91844f1a2a0321fcb95d1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.614031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16dd68db91844f1a2a0321fcb95d1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.614750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16dd68db91844f1a2a0321fcb95d1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.621924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47b063f2-993f-407c-8bde-3a7a12d7e264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16dd68db91844f1a2a0321fcb95d1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:53.628973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 947/1894] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:53 2022] DELETE /identity/v3/users/a16dd68db91844f1a2a0321fcb95d1b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:54.078197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.079701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:54.093082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e6de0-1a63-48af-b47e-53c26face28e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.153161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.154078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.154447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.154805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] PATH_INFO: `/v3/projects/3dc4653d5c264edd8a7a799332618046` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.156526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc4653d5c264edd8a7a799332618046)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.159017 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.198935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:54.216795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc4653d5c264edd8a7a799332618046', 'request_id': 'req-421c7490-cde8-4204-8683-16284223d7b9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:54.222503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc4653d5c264edd8a7a799332618046', 'request_id': 'req-421c7490-cde8-4204-8683-16284223d7b9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:54.229591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc4653d5c264edd8a7a799332618046', 'request_id': 'req-421c7490-cde8-4204-8683-16284223d7b9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:54.244114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] The token cache is being invalidate because project 3dc4653d5c264edd8a7a799332618046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:54.244437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:54.245054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-421c7490-cde8-4204-8683-16284223d7b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:54.249008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 948/1895] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:54 2022] DELETE /identity/v3/projects/3dc4653d5c264edd8a7a799332618046 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:54.254057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e5b2-6984-4ec6-9583-7597631e6268 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.254661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e5b2-6984-4ec6-9583-7597631e6268 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.254923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465e5b2-6984-4ec6-9583-7597631e6268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.276367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c465e5b2-6984-4ec6-9583-7597631e6268 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:54.277578 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c465e5b2-6984-4ec6-9583-7597631e6268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:54.340396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 948/1896] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:54.348439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a394-e30d-4b15-8a64-e1b4b1b02118 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.348863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a394-e30d-4b15-8a64-e1b4b1b02118 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.349284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a394-e30d-4b15-8a64-e1b4b1b02118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.382689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c442a394-e30d-4b15-8a64-e1b4b1b02118 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:54.384514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c442a394-e30d-4b15-8a64-e1b4b1b02118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:54.393921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 949/1897] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:54.398370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e5b2-6984-4ec6-9583-7597631e6268 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.408236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465e5b2-6984-4ec6-9583-7597631e6268 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.413253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.413870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.414149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.414421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.415068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:54.415370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.417076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.429215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 949/1898] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:54.434108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442a394-e30d-4b15-8a64-e1b4b1b02118 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.447529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442a394-e30d-4b15-8a64-e1b4b1b02118 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.453067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.453674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.453952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.454191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.454859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.456820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.490991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 950/1899] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:54 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:54.495573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.508871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1296-a368-4f0b-a0e6-1314f683a66d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.535474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:54.568909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.569652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.569804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.570095 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.570706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.572712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.596600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 950/1900] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:54 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:54.600940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.612683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825029a-8f7c-45e9-856e-4f262a96c534 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.617905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.618515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.618811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.619085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.619733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.621381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.631582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 951/1901] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:54.635666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.646647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac9fa-957c-4b09-964d-2aea056e09d1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.651983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41caf6-1c68-41ba-ac84-89d95ab71d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.652656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41caf6-1c68-41ba-ac84-89d95ab71d27 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.652951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41caf6-1c68-41ba-ac84-89d95ab71d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.653232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41caf6-1c68-41ba-ac84-89d95ab71d27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.653901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41caf6-1c68-41ba-ac84-89d95ab71d27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.655535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41caf6-1c68-41ba-ac84-89d95ab71d27 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.668384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 951/1902] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:54.673041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.685184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8c19a-94f5-4a11-9414-7c6b4bfeea48 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.691125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.691814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.692088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.692391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] PATH_INFO: `/v3/projects/573f0f19fc084f8bb4e3bcef5385463a/users/04085dc328084d868756fe98173cfa1a/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.709445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] RBAC: Authorizing `identity:create_grant(project_id=573f0f19fc084f8bb4e3bcef5385463a, user_id=04085dc328084d868756fe98173cfa1a, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.712131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.728167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 952/1903] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:54 2022] PUT /identity/v3/projects/573f0f19fc084f8bb4e3bcef5385463a/users/04085dc328084d868756fe98173cfa1a/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:54.738643 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e912c-4300-42c3-a0b6-ea64eb715d62 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.738962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e912c-4300-42c3-a0b6-ea64eb715d62 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.739232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e912c-4300-42c3-a0b6-ea64eb715d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.768668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d63e912c-4300-42c3-a0b6-ea64eb715d62 None None] MFA Rules not processed for user `04085dc328084d868756fe98173cfa1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:54.769890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d63e912c-4300-42c3-a0b6-ea64eb715d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:54.838005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 952/1904] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:54.844024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.910541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95347a75-b35c-449f-9d5e-41f04341511c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:54.915538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:54.916177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:54.916854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:54.917127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:54.917759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:54.919673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:54.956037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 953/1905] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:54 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:54.960629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e912c-4300-42c3-a0b6-ea64eb715d62 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:54.972212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e912c-4300-42c3-a0b6-ea64eb715d62 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:55.033890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:55.034646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:55.034927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:55.035203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:55.035848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:55.038283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:55.065563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 953/1906] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:54 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:55.066898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:55.077578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae314bd-12a2-4364-a095-855ca39bff6a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:55.082355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:55.082986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:55.083496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:55.083818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:55.084721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:55.086342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:55.100946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 954/1907] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:55.102751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:55.113253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e84ec-b9ee-4958-8236-05090000dfc7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:55.117906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:55.118574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:55.118851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:55.119122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] PATH_INFO: `/v3/projects/e2f7be8c9db644e08f171a71816d9b17/users/1a28e6c5c85148c6a6ab9b8047ff4a10/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:55.135666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f7be8c9db644e08f171a71816d9b17, user_id=1a28e6c5c85148c6a6ab9b8047ff4a10, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:55.138921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:55.160829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 954/1908] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:55 2022] PUT /identity/v3/projects/e2f7be8c9db644e08f171a71816d9b17/users/1a28e6c5c85148c6a6ab9b8047ff4a10/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:55.162488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:55.172969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c4bd7-cb8d-4bd2-bcf5-c8f7e43155fc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:55.194382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:55.234122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabc27a-856e-41c9-928c-c8a0e8334a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:55.234743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabc27a-856e-41c9-928c-c8a0e8334a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:55.235018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabc27a-856e-41c9-928c-c8a0e8334a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:55.235291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabc27a-856e-41c9-928c-c8a0e8334a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:55.235925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabc27a-856e-41c9-928c-c8a0e8334a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:55.237600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabc27a-856e-41c9-928c-c8a0e8334a67 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:55.248261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 955/1909] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:55.252647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:55.258715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:55.266013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53127385-6eee-4970-b54e-e87fc2e4ec48 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:55.272100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:55.272836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:55.273119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:55.273392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] PATH_INFO: `/v3/projects/e2f7be8c9db644e08f171a71816d9b17/users/1a28e6c5c85148c6a6ab9b8047ff4a10/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:55.275348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f7be8c9db644e08f171a71816d9b17, user_id=1a28e6c5c85148c6a6ab9b8047ff4a10, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:55.278061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:55.292191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 955/1910] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:55 2022] PUT /identity/v3/projects/e2f7be8c9db644e08f171a71816d9b17/users/1a28e6c5c85148c6a6ab9b8047ff4a10/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:55.302490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ba64b-6fee-495c-a61a-1ff387c8bad2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:55.302849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ba64b-6fee-495c-a61a-1ff387c8bad2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:55.303051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ba64b-6fee-495c-a61a-1ff387c8bad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:55.331179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ea4ba64b-6fee-495c-a61a-1ff387c8bad2 None None] MFA Rules not processed for user `1a28e6c5c85148c6a6ab9b8047ff4a10`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:55.332411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea4ba64b-6fee-495c-a61a-1ff387c8bad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:55.404209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 956/1911] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:55.415710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:55.417753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:55.479678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f5383-6524-405b-ae97-2909aa5f26f0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:55.483708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770204f4-e690-479c-a34e-73e5c96b4318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:55.484367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770204f4-e690-479c-a34e-73e5c96b4318 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:55.484632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770204f4-e690-479c-a34e-73e5c96b4318 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:55.484903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770204f4-e690-479c-a34e-73e5c96b4318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:55.494378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770204f4-e690-479c-a34e-73e5c96b4318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:55.495837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770204f4-e690-479c-a34e-73e5c96b4318 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:55.505313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 956/1912] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:55.944682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ba64b-6fee-495c-a61a-1ff387c8bad2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:56.021153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ba64b-6fee-495c-a61a-1ff387c8bad2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:56.027054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:56.027472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:56.027765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:56.028030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] PATH_INFO: `/v3/users/04085dc328084d868756fe98173cfa1a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:56.029266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] RBAC: Authorizing `identity:delete_user(user_id=04085dc328084d868756fe98173cfa1a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:56.031204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:56.083403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04085dc328084d868756fe98173cfa1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.084102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04085dc328084d868756fe98173cfa1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.085815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:56.094215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04085dc328084d868756fe98173cfa1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.107718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04085dc328084d868756fe98173cfa1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.108472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04085dc328084d868756fe98173cfa1a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.121065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 957/1913] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:55 2022] DELETE /identity/v3/users/04085dc328084d868756fe98173cfa1a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:56.125675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:56.125962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:56.126228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:56.146409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:56.147612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:56.213752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 957/1914] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:02:56.223655 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:56.225462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:56.289412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaed28d-cd22-431f-a2b8-6f2dc4ce395e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:56.293312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:56.294033 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:56.294412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:56.294801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:56.310986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:56.310986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:56.319865 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 958/1915] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:02:56.606763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:56.608432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:56.621215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a2199-1d82-4f92-ac37-f9e66f0b59fa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:56.626131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:56.626794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:56.627083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:56.627360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] PATH_INFO: `/v3/projects/573f0f19fc084f8bb4e3bcef5385463a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:56.628640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] RBAC: Authorizing `identity:delete_project(project_id=573f0f19fc084f8bb4e3bcef5385463a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:56.631055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:56.682064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573f0f19fc084f8bb4e3bcef5385463a', 'request_id': 'req-54a6e99f-84c4-4279-babd-afa1ca5c76db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.690119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573f0f19fc084f8bb4e3bcef5385463a', 'request_id': 'req-54a6e99f-84c4-4279-babd-afa1ca5c76db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.696996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573f0f19fc084f8bb4e3bcef5385463a', 'request_id': 'req-54a6e99f-84c4-4279-babd-afa1ca5c76db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.711306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] The token cache is being invalidate because project 573f0f19fc084f8bb4e3bcef5385463a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:56.711630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.712270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.713799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 958/1916] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:56 2022] DELETE /identity/v3/projects/573f0f19fc084f8bb4e3bcef5385463a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:56.718019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:56.728748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f3da-00c3-47bb-95ca-f0f879e64a33 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:56.794892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:56.795532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:56.795837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:56.796088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] PATH_INFO: `/v3/users/1a28e6c5c85148c6a6ab9b8047ff4a10` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:56.797315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a28e6c5c85148c6a6ab9b8047ff4a10)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:56.799309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:56.856238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a28e6c5c85148c6a6ab9b8047ff4a10'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.856890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a28e6c5c85148c6a6ab9b8047ff4a10'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.866197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a28e6c5c85148c6a6ab9b8047ff4a10'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.877297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:56.878698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a28e6c5c85148c6a6ab9b8047ff4a10'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.879374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-decaf2df-dd95-4634-9bf3-699b938c7270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a28e6c5c85148c6a6ab9b8047ff4a10'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:56.893545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 959/1917] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:02:56 2022] DELETE /identity/v3/users/1a28e6c5c85148c6a6ab9b8047ff4a10 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:57.187113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:57.188619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:57.203267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6e99f-84c4-4279-babd-afa1ca5c76db admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:57.216871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:57.290939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:57.291651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.292487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.292487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] PATH_INFO: `/v3/projects/e2f7be8c9db644e08f171a71816d9b17` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.294899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f7be8c9db644e08f171a71816d9b17)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:57.298774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:57.354193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f7be8c9db644e08f171a71816d9b17', 'request_id': 'req-a37ef192-99c8-47ee-a536-031268f2e174'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:57.362295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f7be8c9db644e08f171a71816d9b17', 'request_id': 'req-a37ef192-99c8-47ee-a536-031268f2e174'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:57.374239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f7be8c9db644e08f171a71816d9b17', 'request_id': 'req-a37ef192-99c8-47ee-a536-031268f2e174'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:57.384319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] The token cache is being invalidate because project e2f7be8c9db644e08f171a71816d9b17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:02:57.384705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:57.385343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a37ef192-99c8-47ee-a536-031268f2e174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:02:57.386912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 959/1918] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:02:57 2022] DELETE /identity/v3/projects/e2f7be8c9db644e08f171a71816d9b17 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:57.391974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d36368-6333-4e04-a6b8-d144407b6008 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.392469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d36368-6333-4e04-a6b8-d144407b6008 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.392644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d36368-6333-4e04-a6b8-d144407b6008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.413057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-50d36368-6333-4e04-a6b8-d144407b6008 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:57.414272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-50d36368-6333-4e04-a6b8-d144407b6008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:57.490439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 960/1919] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:57.498127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f542cd2-f5ec-406c-98d1-b45d9811dc36 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.498414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f542cd2-f5ec-406c-98d1-b45d9811dc36 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.498699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f542cd2-f5ec-406c-98d1-b45d9811dc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.518994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0f542cd2-f5ec-406c-98d1-b45d9811dc36 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:57.520355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0f542cd2-f5ec-406c-98d1-b45d9811dc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:57.528925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 960/1920] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:57.533394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d36368-6333-4e04-a6b8-d144407b6008 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:57.543535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d36368-6333-4e04-a6b8-d144407b6008 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:57.548103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:57.548748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.549351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.549351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.549938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:02:57.550235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:57.551871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:57.569084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 961/1921] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:02:57.570272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f542cd2-f5ec-406c-98d1-b45d9811dc36 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:57.580501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f542cd2-f5ec-406c-98d1-b45d9811dc36 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:57.585249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:57.585883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.586160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.586426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.587069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:57.589184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:57.623940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 961/1922] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:57 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:57.625292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:57.636041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca8824-2cd9-4675-aeb2-c8b676ed665b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:57.697422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:57.698051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.698334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.698603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.699235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:57.701255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:57.728706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 962/1923] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:57 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:57.730482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:57.742201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8346a-7ca7-4c95-9163-a7c66ee5d3ea admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:57.749535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:57.750245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.750589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.750923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.752316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:57.754103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:57.769885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 962/1924] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:57.771014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:57.781941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674983f4-523c-4197-a406-0c789d3b3fc7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:57.787436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73956edd-156e-407a-9bc8-0aefe7d6298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:57.788084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73956edd-156e-407a-9bc8-0aefe7d6298d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.788581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73956edd-156e-407a-9bc8-0aefe7d6298d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.788937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73956edd-156e-407a-9bc8-0aefe7d6298d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.789721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73956edd-156e-407a-9bc8-0aefe7d6298d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:57.791284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73956edd-156e-407a-9bc8-0aefe7d6298d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:57.806433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 963/1925] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:02:57.808081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:57.818209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ed0c-face-446f-a60d-c5c902c4b012 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:57.835922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:57.836651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.836929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.837200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] PATH_INFO: `/v3/projects/c3d7200adf6e4a3a8d4d220c187f29e8/users/8594238cf23746a8a5a45b7473e2f883/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.854294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d7200adf6e4a3a8d4d220c187f29e8, user_id=8594238cf23746a8a5a45b7473e2f883, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:57.857074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:57.871945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 963/1926] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:57 2022] PUT /identity/v3/projects/c3d7200adf6e4a3a8d4d220c187f29e8/users/8594238cf23746a8a5a45b7473e2f883/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:02:57.883963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17185-0781-4b6c-96ec-1625cc1b147f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:57.884276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17185-0781-4b6c-96ec-1625cc1b147f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:57.884546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d17185-0781-4b6c-96ec-1625cc1b147f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:57.914518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f3d17185-0781-4b6c-96ec-1625cc1b147f None None] MFA Rules not processed for user `8594238cf23746a8a5a45b7473e2f883`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:57.915721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3d17185-0781-4b6c-96ec-1625cc1b147f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:57.962247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:57.989067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 964/1927] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:57.993831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:58.039843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:58.059135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168ad22-819b-47ea-8a51-89924b09ea34 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:58.064142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:58.064949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.065047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.065467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.066109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:58.068124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:58.097730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 964/1928] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:02:57 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:02:58.101806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17185-0781-4b6c-96ec-1625cc1b147f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:58.112890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d17185-0781-4b6c-96ec-1625cc1b147f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:58.169396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:58.170105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.170378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.170644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.171273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:58.173233 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:58.196832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 965/1929] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:02:58 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:02:58.200973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:58.211645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e385242-b40a-4804-a3f8-1b1310c1b288 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:58.217081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:58.217688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.217962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.218230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.218864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:58.220541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:58.235272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 965/1930] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:58.240199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:58.250953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950df40-bd5a-4d41-8184-4495e80d19cb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:58.255613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:58.256295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.256569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.256875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] PATH_INFO: `/v3/projects/b322ec54abc148ab8ef427cbbcaddfb3/users/d364eb61f11946669e485cedffbde299/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.272612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b322ec54abc148ab8ef427cbbcaddfb3, user_id=d364eb61f11946669e485cedffbde299, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:58.275275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:58.291710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 966/1931] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:58 2022] PUT /identity/v3/projects/b322ec54abc148ab8ef427cbbcaddfb3/users/d364eb61f11946669e485cedffbde299/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:58.295794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:58.307342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c049340-90dc-4f01-8157-73aef032aa0f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:58.372101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8697b-2cfc-4b2a-be57-02e5c3ce2df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:58.372778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8697b-2cfc-4b2a-be57-02e5c3ce2df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.373058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8697b-2cfc-4b2a-be57-02e5c3ce2df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.373351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8697b-2cfc-4b2a-be57-02e5c3ce2df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.376803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8697b-2cfc-4b2a-be57-02e5c3ce2df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:58.378453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8697b-2cfc-4b2a-be57-02e5c3ce2df2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:58.391092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 966/1932] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:02:58.395330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:58.406121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46baaaaa-6d1a-465b-b797-0d3f5578f119 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:58.410929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:58.411622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.411902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.412227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] PATH_INFO: `/v3/projects/b322ec54abc148ab8ef427cbbcaddfb3/users/d364eb61f11946669e485cedffbde299/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.414234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b322ec54abc148ab8ef427cbbcaddfb3, user_id=d364eb61f11946669e485cedffbde299, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:58.416920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:58.427752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 967/1933] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:02:58 2022] PUT /identity/v3/projects/b322ec54abc148ab8ef427cbbcaddfb3/users/d364eb61f11946669e485cedffbde299/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:02:58.438126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.438417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.438691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.468941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] MFA Rules not processed for user `d364eb61f11946669e485cedffbde299`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:02:58.470140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:58.543098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 967/1934] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:02:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:02:58.553843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:02:58.555567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:02:58.613517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f2c75-97a1-4cc4-a214-4466e13783fd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:02:58.617692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf6a1a-2073-468b-b94b-645397ed474e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:02:58.618303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6a1a-2073-468b-b94b-645397ed474e service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:02:58.618584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6a1a-2073-468b-b94b-645397ed474e service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:02:58.618859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6a1a-2073-468b-b94b-645397ed474e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:02:58.670563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:02:58.687589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf6a1a-2073-468b-b94b-645397ed474e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:02:58.689894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf6a1a-2073-468b-b94b-645397ed474e service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:02:58.703202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 968/1935] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:02:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:08.540577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:08.698478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:15.190851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:15.192891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:15.260961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885aa802-3dab-42ad-808a-07b68fd7d682 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:15.265950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:15.266623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:15.266905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:15.267181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] PATH_INFO: `/v3/users/eb81051fdb914d6abd9f5f2db542e564` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:15.268365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb81051fdb914d6abd9f5f2db542e564)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:15.270327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:15.320064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb81051fdb914d6abd9f5f2db542e564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.328393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb81051fdb914d6abd9f5f2db542e564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.341276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb81051fdb914d6abd9f5f2db542e564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.341945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb81051fdb914d6abd9f5f2db542e564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.349038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb81051fdb914d6abd9f5f2db542e564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.355662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 968/1936] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:15 2022] DELETE /identity/v3/users/eb81051fdb914d6abd9f5f2db542e564 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:15.360411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:15.360939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:15.360939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:15.380420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:15.381629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:15.446299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 969/1937] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:15.456229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:15.457743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:15.518552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa81bc4-936d-4676-99a8-3fd7a8f58fba admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:15.522591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:15.523260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:15.523989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:15.524308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:15.534781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:15.536268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:15.546587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 969/1938] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:15.828192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:15.829841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:15.845307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0058bfa-029d-4690-9c73-f32d7f145b39 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:15.856341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:15.857177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:15.857451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:15.857709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] PATH_INFO: `/v3/projects/76544b1e49f748bcbd66fbb7d459f7d5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:15.859098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76544b1e49f748bcbd66fbb7d459f7d5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:15.861125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:15.914476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76544b1e49f748bcbd66fbb7d459f7d5', 'request_id': 'req-e314753a-b5f4-4504-afab-a5e34492279b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.920556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76544b1e49f748bcbd66fbb7d459f7d5', 'request_id': 'req-e314753a-b5f4-4504-afab-a5e34492279b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.927105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76544b1e49f748bcbd66fbb7d459f7d5', 'request_id': 'req-e314753a-b5f4-4504-afab-a5e34492279b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.942962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] The token cache is being invalidate because project 76544b1e49f748bcbd66fbb7d459f7d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:15.943337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.944107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:15.945916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 970/1939] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:15 2022] DELETE /identity/v3/projects/76544b1e49f748bcbd66fbb7d459f7d5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:15.949898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:15.955137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:15.956522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:15.960067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddb772-4e65-493c-b5ec-5183930b9c14 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:15.979531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:16.017870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314753a-b5f4-4504-afab-a5e34492279b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:16.021275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:16.021993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.022107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:16.022417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.022671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.022767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] PATH_INFO: `/v3/users/475ff79ea3954154965255915aa45420` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.022988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.023438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.023955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=475ff79ea3954154965255915aa45420)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:16.025474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:16.025967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:16.069896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:16.078510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '475ff79ea3954154965255915aa45420'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.087312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '475ff79ea3954154965255915aa45420'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.097910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:16.099453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:16.099589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '475ff79ea3954154965255915aa45420'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.100267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '475ff79ea3954154965255915aa45420'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.108566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-628b39ba-65af-4942-bb08-a361648728fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '475ff79ea3954154965255915aa45420'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.110274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 971/1940] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:16.114491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 970/1941] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:15 2022] DELETE /identity/v3/users/475ff79ea3954154965255915aa45420 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:16.434428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:16.436725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:16.456360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a726cb-8b3e-4d8a-aa6a-dc0b32ce2560 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:16.548357 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:16.549026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.549301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.549596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] PATH_INFO: `/v3/projects/bf32c6c197874ff7bbbfdfb1e3d354d8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.550750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf32c6c197874ff7bbbfdfb1e3d354d8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:16.552714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:16.605680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf32c6c197874ff7bbbfdfb1e3d354d8', 'request_id': 'req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.611101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf32c6c197874ff7bbbfdfb1e3d354d8', 'request_id': 'req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.617277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf32c6c197874ff7bbbfdfb1e3d354d8', 'request_id': 'req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.631687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] The token cache is being invalidate because project bf32c6c197874ff7bbbfdfb1e3d354d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:16.631982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.632907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4be033f4-5ac0-4fb4-87d3-a12c3a6dab28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:16.634489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 972/1942] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:16 2022] DELETE /identity/v3/projects/bf32c6c197874ff7bbbfdfb1e3d354d8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:16.639623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea2f7d-27cc-4f90-a107-150f452860fe None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.640139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea2f7d-27cc-4f90-a107-150f452860fe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.640139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea2f7d-27cc-4f90-a107-150f452860fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.659986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9bea2f7d-27cc-4f90-a107-150f452860fe None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:16.661260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9bea2f7d-27cc-4f90-a107-150f452860fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:16.733382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 971/1943] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:16.741811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499bbde-1afb-437b-8ae6-eec326b5e103 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.742297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499bbde-1afb-437b-8ae6-eec326b5e103 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.742582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499bbde-1afb-437b-8ae6-eec326b5e103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.764109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4499bbde-1afb-437b-8ae6-eec326b5e103 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:16.765362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4499bbde-1afb-437b-8ae6-eec326b5e103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:16.774069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 973/1944] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:16.778670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea2f7d-27cc-4f90-a107-150f452860fe None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:16.788949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea2f7d-27cc-4f90-a107-150f452860fe None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:16.794896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:16.795521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.795803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.796070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.796765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:16.797063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:16.798713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:16.813412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 972/1945] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:16.817803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499bbde-1afb-437b-8ae6-eec326b5e103 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:16.827226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499bbde-1afb-437b-8ae6-eec326b5e103 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:16.832301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:16.832897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.833170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.833435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.834070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:16.836127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:16.868248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 974/1946] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:16 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:16.873044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:16.883942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce16cc-4885-4441-ab6e-4eb6ce71bb1d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:16.911115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:16.943280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:16.943895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.944256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.944542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.945251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:16.947188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:16.971609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 973/1947] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:16 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:16.976174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:16.988501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445ae7d-d43c-46ac-8639-5a8cda04e8d6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:16.994185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:16.994797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:16.995089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:16.995335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:16.996000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:16.997818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.008943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 975/1948] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:17.013106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.024286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d61f35-dcef-433a-b80b-e5e1e3a8e303 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.030235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3efeb-e2f5-4753-9d4a-7980a8f95221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.030848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3efeb-e2f5-4753-9d4a-7980a8f95221 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.031128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3efeb-e2f5-4753-9d4a-7980a8f95221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.031404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3efeb-e2f5-4753-9d4a-7980a8f95221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.032041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3efeb-e2f5-4753-9d4a-7980a8f95221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.033733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb3efeb-e2f5-4753-9d4a-7980a8f95221 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.044272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 974/1949] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:17.048487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.062120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba2beb-50e9-4967-a946-70b4062bbadb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.071095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.072084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.072426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.072905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] PATH_INFO: `/v3/projects/0bb818471b254991b0bbef187fdbf5dc/users/a11bb6c37e57473a87c4f56083ab63ff/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.093739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bb818471b254991b0bbef187fdbf5dc, user_id=a11bb6c37e57473a87c4f56083ab63ff, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.096489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.108650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:17.113958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 976/1950] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:17 2022] PUT /identity/v3/projects/0bb818471b254991b0bbef187fdbf5dc/users/a11bb6c37e57473a87c4f56083ab63ff/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:17.126405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342f7d-8651-4948-9673-c8f7ad613fe5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.126748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342f7d-8651-4948-9673-c8f7ad613fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.127024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a342f7d-8651-4948-9673-c8f7ad613fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.156792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6a342f7d-8651-4948-9673-c8f7ad613fe5 None None] MFA Rules not processed for user `a11bb6c37e57473a87c4f56083ab63ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:17.158015 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6a342f7d-8651-4948-9673-c8f7ad613fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:17.234029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 975/1951] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:17.238969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.307311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c33ad2-e0c3-4aac-88f2-6d6593603d8a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.316142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.317165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.317600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.317988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.318965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.322323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.356521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 977/1952] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:17 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:17.360867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342f7d-8651-4948-9673-c8f7ad613fe5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.372169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a342f7d-8651-4948-9673-c8f7ad613fe5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.430022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.430850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.431358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.431824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.432682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.435710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.460379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 976/1953] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:17 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:17.464555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.478310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe109b-5aaf-4c78-bb3d-8e8b92f80859 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.484180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.485054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.485338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.485722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.486550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.488335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.502100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 978/1954] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:17.507060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.517454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6f123-fc1c-4389-aa11-442336620c71 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.522203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.522869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.523147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.523418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] PATH_INFO: `/v3/projects/9e27c4ec896844d6886de69642ec030a/users/d093b04758ee4433aa79ecb81ef5d4bc/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.538904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e27c4ec896844d6886de69642ec030a, user_id=d093b04758ee4433aa79ecb81ef5d4bc, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.541843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.556600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 977/1955] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:17 2022] PUT /identity/v3/projects/9e27c4ec896844d6886de69642ec030a/users/d093b04758ee4433aa79ecb81ef5d4bc/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:17.561363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.573556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938af529-f78c-4b3d-a1ee-843ce0f2bb0c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.630492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977023b-95a0-4d2b-a240-5112b64235f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.631106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977023b-95a0-4d2b-a240-5112b64235f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.631383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977023b-95a0-4d2b-a240-5112b64235f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.631653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977023b-95a0-4d2b-a240-5112b64235f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.632311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977023b-95a0-4d2b-a240-5112b64235f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.634013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977023b-95a0-4d2b-a240-5112b64235f7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.644712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 979/1956] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:17.648893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.654571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:17.661975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56aa2-e74e-4fec-be28-29ba313cc2c3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.667581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.668264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.668581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.668851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] PATH_INFO: `/v3/projects/9e27c4ec896844d6886de69642ec030a/users/d093b04758ee4433aa79ecb81ef5d4bc/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.670823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e27c4ec896844d6886de69642ec030a, user_id=d093b04758ee4433aa79ecb81ef5d4bc, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.673568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.685118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 978/1957] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:17 2022] PUT /identity/v3/projects/9e27c4ec896844d6886de69642ec030a/users/d093b04758ee4433aa79ecb81ef5d4bc/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:17.695278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bffa1-dc07-466c-addf-82f4143ac9b7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.695562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bffa1-dc07-466c-addf-82f4143ac9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.695837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bffa1-dc07-466c-addf-82f4143ac9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.724055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7c0bffa1-dc07-466c-addf-82f4143ac9b7 None None] MFA Rules not processed for user `d093b04758ee4433aa79ecb81ef5d4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:17.725350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c0bffa1-dc07-466c-addf-82f4143ac9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:17.785209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:17.794627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 980/1958] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:17.806245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.808239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:17.871026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c58ad4-1cad-4183-bb88-6e41b16dec24 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.875038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55fb77-979b-44ca-882e-9b5ef082da59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.875678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55fb77-979b-44ca-882e-9b5ef082da59 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.875950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55fb77-979b-44ca-882e-9b5ef082da59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.876250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55fb77-979b-44ca-882e-9b5ef082da59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.886529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55fb77-979b-44ca-882e-9b5ef082da59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:17.887962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55fb77-979b-44ca-882e-9b5ef082da59 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:17.897838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 979/1959] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:17.926794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bffa1-dc07-466c-addf-82f4143ac9b7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:17.991794 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bffa1-dc07-466c-addf-82f4143ac9b7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:17.997224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:17.997893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:17.998179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:17.998759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] PATH_INFO: `/v3/users/a11bb6c37e57473a87c4f56083ab63ff` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:17.999637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11bb6c37e57473a87c4f56083ab63ff)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:18.001683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:18.048572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11bb6c37e57473a87c4f56083ab63ff'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.049239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11bb6c37e57473a87c4f56083ab63ff'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.058273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11bb6c37e57473a87c4f56083ab63ff'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.074370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11bb6c37e57473a87c4f56083ab63ff'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.075257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11bb6c37e57473a87c4f56083ab63ff'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.088260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 981/1960] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:17 2022] DELETE /identity/v3/users/a11bb6c37e57473a87c4f56083ab63ff => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:18.092890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:18.093198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:18.093467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:18.113801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:18.115005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:18.183033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 980/1961] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:18.196098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:18.198158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:18.265370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c814f-e851-4c32-a9d5-68d7e37a5138 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:18.269629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:18.270232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:18.270506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:18.270777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:18.281100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:18.282652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:18.294070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 982/1962] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:18.581619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:18.583043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:18.598445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181cf12-8da0-40d4-b940-155cc542756c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:18.604876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:18.605572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:18.606215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:18.606494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] PATH_INFO: `/v3/projects/0bb818471b254991b0bbef187fdbf5dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:18.607985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bb818471b254991b0bbef187fdbf5dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:18.610106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:18.664017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bb818471b254991b0bbef187fdbf5dc', 'request_id': 'req-1800722b-9e63-4933-8fc9-c7e84f47f6ae'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.672173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bb818471b254991b0bbef187fdbf5dc', 'request_id': 'req-1800722b-9e63-4933-8fc9-c7e84f47f6ae'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.677964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bb818471b254991b0bbef187fdbf5dc', 'request_id': 'req-1800722b-9e63-4933-8fc9-c7e84f47f6ae'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.692960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] The token cache is being invalidate because project 0bb818471b254991b0bbef187fdbf5dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:18.693247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.693928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.695567 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 981/1963] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:18 2022] DELETE /identity/v3/projects/0bb818471b254991b0bbef187fdbf5dc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:18.707270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:18.717231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddca03-fa75-4530-af0f-e831dc25d6b7 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:18.731102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:18.781414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:18.782050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:18.782327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:18.782595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] PATH_INFO: `/v3/users/d093b04758ee4433aa79ecb81ef5d4bc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:18.783754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d093b04758ee4433aa79ecb81ef5d4bc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:18.785768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:18.837078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd093b04758ee4433aa79ecb81ef5d4bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.837725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd093b04758ee4433aa79ecb81ef5d4bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.846706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd093b04758ee4433aa79ecb81ef5d4bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.859875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd093b04758ee4433aa79ecb81ef5d4bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.860558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a326f5f-cf29-4062-bd8d-41cee62695f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd093b04758ee4433aa79ecb81ef5d4bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:18.873654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 983/1964] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:18 2022] DELETE /identity/v3/users/d093b04758ee4433aa79ecb81ef5d4bc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:19.161147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.162757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:19.177695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800722b-9e63-4933-8fc9-c7e84f47f6ae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.191521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:19.242615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.243371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.243648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.243887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] PATH_INFO: `/v3/projects/9e27c4ec896844d6886de69642ec030a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.245137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e27c4ec896844d6886de69642ec030a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.247076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:19.299424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e27c4ec896844d6886de69642ec030a', 'request_id': 'req-7ba19fc0-aa18-487e-93f5-fb64412ec456'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:19.305947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e27c4ec896844d6886de69642ec030a', 'request_id': 'req-7ba19fc0-aa18-487e-93f5-fb64412ec456'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:19.312274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e27c4ec896844d6886de69642ec030a', 'request_id': 'req-7ba19fc0-aa18-487e-93f5-fb64412ec456'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:19.326862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] The token cache is being invalidate because project 9e27c4ec896844d6886de69642ec030a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:19.327193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:19.327838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7ba19fc0-aa18-487e-93f5-fb64412ec456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:19.329348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 982/1965] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:19 2022] DELETE /identity/v3/projects/9e27c4ec896844d6886de69642ec030a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:19.334328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ffce7-0853-42fe-93ce-f303af52777c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.334648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ffce7-0853-42fe-93ce-f303af52777c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.334914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ffce7-0853-42fe-93ce-f303af52777c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.358486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e07ffce7-0853-42fe-93ce-f303af52777c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:19.359707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e07ffce7-0853-42fe-93ce-f303af52777c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:19.424622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 984/1966] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:19.432626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b874-a37b-4229-8f95-f65d3536b741 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.432907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b874-a37b-4229-8f95-f65d3536b741 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.433179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7b874-a37b-4229-8f95-f65d3536b741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.452850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0ca7b874-a37b-4229-8f95-f65d3536b741 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:19.454044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0ca7b874-a37b-4229-8f95-f65d3536b741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:19.462779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 983/1967] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:19.467043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ffce7-0853-42fe-93ce-f303af52777c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.477539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ffce7-0853-42fe-93ce-f303af52777c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.483120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.483720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.484013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.484308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.484959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:19.485262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.486877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:19.499093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 985/1968] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:19.503601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7b874-a37b-4229-8f95-f65d3536b741 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.513657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7b874-a37b-4229-8f95-f65d3536b741 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.519140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.519743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.520021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.520334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.520963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.522881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:19.559103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 984/1969] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:19 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:19.562980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.574784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab896aba-6482-4052-bfea-f208c02037c2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.598670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:19.640216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.640840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.641121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.641537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.642214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.644156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:19.667638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 986/1970] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:19 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:19.671931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.683214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbce1e9-e396-479a-949b-697836a051cb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.688023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.688663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.688944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.689218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.689853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.691497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:19.702559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 985/1971] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:03:19.706770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.718930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f8296-1b6f-4729-b424-f869d93eef9c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.725996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a5703-78ca-4396-ac27-b3f675cd997f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.726851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5703-78ca-4396-ac27-b3f675cd997f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.727939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5703-78ca-4396-ac27-b3f675cd997f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.727939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a5703-78ca-4396-ac27-b3f675cd997f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.728509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5703-78ca-4396-ac27-b3f675cd997f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.730127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a5703-78ca-4396-ac27-b3f675cd997f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:19.740125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 987/1972] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:03:19.744298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.754488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2cfdc-216f-4d85-828e-a81488da7e52 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.759510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.760167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.760471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.760747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] PATH_INFO: `/v3/projects/1c0a7236551541d3beaef2243709ed2c/users/bdc0e5ebef5b44dfb01105b0d3f08f80/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.776562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0a7236551541d3beaef2243709ed2c, user_id=bdc0e5ebef5b44dfb01105b0d3f08f80, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.779595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:19.795249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 986/1973] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:19 2022] PUT /identity/v3/projects/1c0a7236551541d3beaef2243709ed2c/users/bdc0e5ebef5b44dfb01105b0d3f08f80/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:19.806989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e6f6-19cd-4720-948f-8478a0225194 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.807304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e6f6-19cd-4720-948f-8478a0225194 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.807576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e6f6-19cd-4720-948f-8478a0225194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.835620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9f01e6f6-19cd-4720-948f-8478a0225194 None None] MFA Rules not processed for user `bdc0e5ebef5b44dfb01105b0d3f08f80`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:19.837004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9f01e6f6-19cd-4720-948f-8478a0225194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:19.902745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 988/1974] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:19 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:19.907389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:19.957368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:19.978543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29ee7a-7efb-4ace-b886-33854c965d54 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:19.983641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:19.984299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:19.984728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:19.985016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:19.985669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:19.987603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.016828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 987/1975] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:19 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:20.020971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e6f6-19cd-4720-948f-8478a0225194 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:20.032614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e6f6-19cd-4720-948f-8478a0225194 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:20.121543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:20.122250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.122587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.122915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.123623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:20.125811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.154879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 989/1976] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:20 2022] POST /identity/v3/users => generated 306 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:20.158839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:20.169510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eb98ac-699b-4690-bb7c-513ba33ecd5f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:20.174556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:20.175171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.175458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.175731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.176433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:20.178102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.189463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 988/1977] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:20.193837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:20.205282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f58e95-0ed1-4978-960c-34e4566d5932 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:20.210238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:20.210902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.211184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.211473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] PATH_INFO: `/v3/projects/f4d56bdbb2d147a5b71ea2c6863e7b9f/users/0869261c946f47359209196f67f6dd78/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.226335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d56bdbb2d147a5b71ea2c6863e7b9f, user_id=0869261c946f47359209196f67f6dd78, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:20.229516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.240853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:20.247959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 990/1978] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:20 2022] PUT /identity/v3/projects/f4d56bdbb2d147a5b71ea2c6863e7b9f/users/0869261c946f47359209196f67f6dd78/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:20.253171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:20.265806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928a2d8-b702-460f-9e6f-46851e7831e6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:20.327056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a6255-4806-4d32-94aa-f2e76849376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:20.327655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a6255-4806-4d32-94aa-f2e76849376c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.327931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a6255-4806-4d32-94aa-f2e76849376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.328227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a6255-4806-4d32-94aa-f2e76849376c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.328858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a6255-4806-4d32-94aa-f2e76849376c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:20.330464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a6255-4806-4d32-94aa-f2e76849376c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.340578 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 989/1979] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:20.344846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:20.356441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e246c-f08b-4a02-bed8-b79d17c6d0e6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:20.361163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:20.361857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.362129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.362398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] PATH_INFO: `/v3/projects/f4d56bdbb2d147a5b71ea2c6863e7b9f/users/0869261c946f47359209196f67f6dd78/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.364348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d56bdbb2d147a5b71ea2c6863e7b9f, user_id=0869261c946f47359209196f67f6dd78, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:20.366980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.388466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 991/1980] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:20 2022] PUT /identity/v3/projects/f4d56bdbb2d147a5b71ea2c6863e7b9f/users/0869261c946f47359209196f67f6dd78/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:20.399133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27214eab-885a-4c11-8c2e-f4b3ccde8b45 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.399420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27214eab-885a-4c11-8c2e-f4b3ccde8b45 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.399695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27214eab-885a-4c11-8c2e-f4b3ccde8b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.434516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-27214eab-885a-4c11-8c2e-f4b3ccde8b45 None None] MFA Rules not processed for user `0869261c946f47359209196f67f6dd78`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:20.435755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-27214eab-885a-4c11-8c2e-f4b3ccde8b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:20.510939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 990/1981] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:20 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:20.522013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:20.524267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:20.586603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434de554-edae-4750-a065-91d305a558e1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:20.591634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:20.592200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.592480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.592774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.603247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:20.605403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.619885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 992/1982] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:20.804265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27214eab-885a-4c11-8c2e-f4b3ccde8b45 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:20.814831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27214eab-885a-4c11-8c2e-f4b3ccde8b45 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:20.819126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143348e-d592-4fa8-a8d8-185efbc1017e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:20.819737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143348e-d592-4fa8-a8d8-185efbc1017e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:20.820014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143348e-d592-4fa8-a8d8-185efbc1017e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:20.820328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143348e-d592-4fa8-a8d8-185efbc1017e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:20.854990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:20.878497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143348e-d592-4fa8-a8d8-185efbc1017e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:20.879941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143348e-d592-4fa8-a8d8-185efbc1017e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:20.889060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 991/1983] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:21.741787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:21.743807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:21.811910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876040a4-a7b9-4d29-947a-d956df816b46 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:21.816699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:21.817337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:21.817600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:21.818036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] PATH_INFO: `/v3/users/c252f69aaf234d018ac99edefa3246d7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:21.819180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c252f69aaf234d018ac99edefa3246d7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:21.821234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:21.871190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c252f69aaf234d018ac99edefa3246d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:21.871834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c252f69aaf234d018ac99edefa3246d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:21.881697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c252f69aaf234d018ac99edefa3246d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:21.894134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c252f69aaf234d018ac99edefa3246d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:21.894914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c252f69aaf234d018ac99edefa3246d7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:21.908040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 993/1984] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:21 2022] DELETE /identity/v3/users/c252f69aaf234d018ac99edefa3246d7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:21.912983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:21.913517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:21.913625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:21.937598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:21.938799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:22.005533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 992/1985] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:22.015537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:22.017206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:22.027210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:22.116935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e6512-6518-47b1-980a-ce31553ff452 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:22.122550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:22.123184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:22.123534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:22.123726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:22.135966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:22.137793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:22.149636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 994/1986] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:22.542563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:22.544377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:22.569738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255580d-91c6-4b08-8bff-70700ebdbe64 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:22.575697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:22.576670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:22.576901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:22.577184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] PATH_INFO: `/v3/projects/33921d60041e46b89dbfd9e110ce17a5` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:22.578352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33921d60041e46b89dbfd9e110ce17a5)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:22.580426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:22.628653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33921d60041e46b89dbfd9e110ce17a5', 'request_id': 'req-2ab00101-fb57-4a95-a964-9a3808c81b56'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.635079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33921d60041e46b89dbfd9e110ce17a5', 'request_id': 'req-2ab00101-fb57-4a95-a964-9a3808c81b56'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.640739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33921d60041e46b89dbfd9e110ce17a5', 'request_id': 'req-2ab00101-fb57-4a95-a964-9a3808c81b56'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.654728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] The token cache is being invalidate because project 33921d60041e46b89dbfd9e110ce17a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:22.655018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.655732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.657413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 993/1987] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:22 2022] DELETE /identity/v3/projects/33921d60041e46b89dbfd9e110ce17a5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:22.661606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:22.672004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2964-1f8e-4c29-85bc-eb8be538d7c9 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:22.732911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:22.733666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:22.733946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:22.734215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] PATH_INFO: `/v3/users/2c0e4fb01b2a4c5fbbceb080b2551c76` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:22.735408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0e4fb01b2a4c5fbbceb080b2551c76)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:22.737320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:22.789121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0e4fb01b2a4c5fbbceb080b2551c76'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.789830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0e4fb01b2a4c5fbbceb080b2551c76'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.798660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0e4fb01b2a4c5fbbceb080b2551c76'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.811064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0e4fb01b2a4c5fbbceb080b2551c76'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.811775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bef006dc-25fe-41b3-8ead-cd04675764f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0e4fb01b2a4c5fbbceb080b2551c76'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:22.824879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 995/1988] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:22 2022] DELETE /identity/v3/users/2c0e4fb01b2a4c5fbbceb080b2551c76 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:23.113132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.116760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:23.134184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab00101-fb57-4a95-a964-9a3808c81b56 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.196446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.197094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.197376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.197656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] PATH_INFO: `/v3/projects/14be4cc730ca40f683bd64865cf8236d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.198853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=14be4cc730ca40f683bd64865cf8236d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.200814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:23.202707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:23.255480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14be4cc730ca40f683bd64865cf8236d', 'request_id': 'req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:23.261770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14be4cc730ca40f683bd64865cf8236d', 'request_id': 'req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:23.267683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14be4cc730ca40f683bd64865cf8236d', 'request_id': 'req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:23.280954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] The token cache is being invalidate because project 14be4cc730ca40f683bd64865cf8236d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:23.281247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:23.281983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1fad4431-ae2e-49d2-bdbd-2e985aa1c55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:23.284222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 994/1989] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:23 2022] DELETE /identity/v3/projects/14be4cc730ca40f683bd64865cf8236d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:23.289327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3bd2e-fc18-4d50-92ef-209e1c282011 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.289661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3bd2e-fc18-4d50-92ef-209e1c282011 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.290108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3bd2e-fc18-4d50-92ef-209e1c282011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.312826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e2d3bd2e-fc18-4d50-92ef-209e1c282011 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:23.314143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e2d3bd2e-fc18-4d50-92ef-209e1c282011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:23.341352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:23.381040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 996/1990] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:23.389349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa400f1e-6510-49b2-8c3c-58ef9246619c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.389636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa400f1e-6510-49b2-8c3c-58ef9246619c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.389950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa400f1e-6510-49b2-8c3c-58ef9246619c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.411371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fa400f1e-6510-49b2-8c3c-58ef9246619c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:23.412841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fa400f1e-6510-49b2-8c3c-58ef9246619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:23.420851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 995/1991] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:23.425603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3bd2e-fc18-4d50-92ef-209e1c282011 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.437054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3bd2e-fc18-4d50-92ef-209e1c282011 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.442897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.443656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.443972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.444272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.445037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:23.445348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.447250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:23.463734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 997/1992] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:03:23.465495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa400f1e-6510-49b2-8c3c-58ef9246619c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.475348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa400f1e-6510-49b2-8c3c-58ef9246619c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.481113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.481782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.482046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.482328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.482991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.485277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:23.518371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 996/1993] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:23 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:23.536421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.536421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db5402-8b47-4b78-8514-99e4894e1f5f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.615698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.616732 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.617361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.617811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.618693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.622504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:23.658476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 998/1994] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:23 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:23.660527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.672346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f71041-6ee0-4023-a8fa-66adb3b6b9dc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.677233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.677859 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.678126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.678401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.679431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.680997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:23.696200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 997/1995] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:23.698434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.710947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7cbb-f060-4a87-b2c1-042b4cc50a4b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.715985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec88160-bca6-4000-9289-f8581de1ab0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.716658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88160-bca6-4000-9289-f8581de1ab0b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.716926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88160-bca6-4000-9289-f8581de1ab0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.717223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec88160-bca6-4000-9289-f8581de1ab0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.717875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88160-bca6-4000-9289-f8581de1ab0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.719481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec88160-bca6-4000-9289-f8581de1ab0b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:23.733205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 999/1996] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:23.737499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.749410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d6baa-8c54-456b-b6aa-5b948f8e5179 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.754786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.755600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.755939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.756289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] PATH_INFO: `/v3/projects/1bbb7453668f40bd91dad732793dc232/users/ad3a55b5c81045a788202adc0f5d53d3/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.779009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbb7453668f40bd91dad732793dc232, user_id=ad3a55b5c81045a788202adc0f5d53d3, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.781965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:23.796724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 998/1997] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:23 2022] PUT /identity/v3/projects/1bbb7453668f40bd91dad732793dc232/users/ad3a55b5c81045a788202adc0f5d53d3/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:23.809008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518859a7-0460-405d-a493-5831fe2e9618 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.809548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518859a7-0460-405d-a493-5831fe2e9618 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.809548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518859a7-0460-405d-a493-5831fe2e9618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.838838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-518859a7-0460-405d-a493-5831fe2e9618 None None] MFA Rules not processed for user `ad3a55b5c81045a788202adc0f5d53d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:23.839986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-518859a7-0460-405d-a493-5831fe2e9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:23.905700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1000/1998] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:23.910507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:23.975838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0987d62-4425-4746-8ab9-1f73621a23b5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:23.982668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:23.983280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:23.983560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:23.983825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:23.984524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:23.986461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.016801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 999/1999] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:23 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:24.021103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518859a7-0460-405d-a493-5831fe2e9618 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.033024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518859a7-0460-405d-a493-5831fe2e9618 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:24.054081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:24.092992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:24.093616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.093999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.094253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.094978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:24.096938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.119169 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1001/2000] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:24 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:24.124611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.137982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019c4bb-3674-47d6-8f4b-6158c0c8511a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:24.143918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:24.144749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.145035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.145315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.145979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:24.147637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.153862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:24.160712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1000/2001] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:24.165576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.176003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31e9d1-4aeb-490b-b21a-e7da320dd8bc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:24.181508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:24.182180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.182485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.182753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] PATH_INFO: `/v3/projects/889914e46a7e4d6186d179de67fe9b22/users/7b192eacd6204f5c83ad63291dfcb04b/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.197992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=889914e46a7e4d6186d179de67fe9b22, user_id=7b192eacd6204f5c83ad63291dfcb04b, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:24.200730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.215301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1002/2002] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:24 2022] PUT /identity/v3/projects/889914e46a7e4d6186d179de67fe9b22/users/7b192eacd6204f5c83ad63291dfcb04b/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:24.219645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.230317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679966b-af6a-4089-9b53-54815908c225 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:24.288207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad189cc4-6e67-4823-859b-f67444705b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:24.288846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189cc4-6e67-4823-859b-f67444705b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.289128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189cc4-6e67-4823-859b-f67444705b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.289401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad189cc4-6e67-4823-859b-f67444705b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.290043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad189cc4-6e67-4823-859b-f67444705b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:24.291752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad189cc4-6e67-4823-859b-f67444705b7b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.301967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1001/2003] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:24.306565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.317089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717224fb-4a65-4f54-9af2-51e27acf11e5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:24.322277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:24.322939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.323216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.323488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] PATH_INFO: `/v3/projects/889914e46a7e4d6186d179de67fe9b22/users/7b192eacd6204f5c83ad63291dfcb04b/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.325418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] RBAC: Authorizing `identity:create_grant(project_id=889914e46a7e4d6186d179de67fe9b22, user_id=7b192eacd6204f5c83ad63291dfcb04b, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:24.328049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.339210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1003/2004] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:24 2022] PUT /identity/v3/projects/889914e46a7e4d6186d179de67fe9b22/users/7b192eacd6204f5c83ad63291dfcb04b/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:24.350606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f345d-91ec-4919-8c88-6b9270c3698a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.350964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f345d-91ec-4919-8c88-6b9270c3698a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.351356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f345d-91ec-4919-8c88-6b9270c3698a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.384192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3c9f345d-91ec-4919-8c88-6b9270c3698a None None] MFA Rules not processed for user `7b192eacd6204f5c83ad63291dfcb04b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:24.385481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c9f345d-91ec-4919-8c88-6b9270c3698a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:24.460783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1002/2005] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:24.472292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.474089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:24.535766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1297d2-1f51-4866-b6f6-4e851c03178e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:24.539789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae113c-8be2-4789-a4aa-a8b430ca6a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:24.540477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae113c-8be2-4789-a4aa-a8b430ca6a3f service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.540757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae113c-8be2-4789-a4aa-a8b430ca6a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.541022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae113c-8be2-4789-a4aa-a8b430ca6a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.551482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae113c-8be2-4789-a4aa-a8b430ca6a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:24.553105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae113c-8be2-4789-a4aa-a8b430ca6a3f service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.564594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1004/2006] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:24.644607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f345d-91ec-4919-8c88-6b9270c3698a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.725069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f345d-91ec-4919-8c88-6b9270c3698a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:24.730464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:24.731236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.731617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.732044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] PATH_INFO: `/v3/users/ad3a55b5c81045a788202adc0f5d53d3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.733298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3a55b5c81045a788202adc0f5d53d3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:24.735521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:24.761363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:24.796979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3a55b5c81045a788202adc0f5d53d3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:24.808981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3a55b5c81045a788202adc0f5d53d3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:24.823496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3a55b5c81045a788202adc0f5d53d3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:24.824246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3a55b5c81045a788202adc0f5d53d3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:24.831894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3a55b5c81045a788202adc0f5d53d3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:24.840896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1003/2007] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:24 2022] DELETE /identity/v3/users/ad3a55b5c81045a788202adc0f5d53d3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:24.845332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:24.845825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:24.845918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:24.866635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:24.867890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:24.950391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1005/2008] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:24.961356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:24.963288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:25.034531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495c7e1-bbea-49d9-ba27-ce64b6a077e7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:25.038486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:25.039121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:25.039404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:25.039710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:25.049324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:25.050779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:25.060770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1004/2009] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:25.341726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:25.343367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:25.360065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7bf4d-7e00-40a6-81e4-43e84905bd98 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:25.365157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:25.365780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:25.366237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:25.366500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] PATH_INFO: `/v3/projects/1bbb7453668f40bd91dad732793dc232` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:25.374797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbb7453668f40bd91dad732793dc232)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:25.376886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:25.396356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:25.442554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbb7453668f40bd91dad732793dc232', 'request_id': 'req-d47d1682-136c-42df-95e8-998cf89f0e0a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.451552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbb7453668f40bd91dad732793dc232', 'request_id': 'req-d47d1682-136c-42df-95e8-998cf89f0e0a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.461000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbb7453668f40bd91dad732793dc232', 'request_id': 'req-d47d1682-136c-42df-95e8-998cf89f0e0a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.476401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] The token cache is being invalidate because project 1bbb7453668f40bd91dad732793dc232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:25.476739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.477869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.492365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1006/2010] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:25 2022] DELETE /identity/v3/projects/1bbb7453668f40bd91dad732793dc232 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:25.492585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:25.494986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8dd2-656e-4a15-8d36-c24d70e7c6d7 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:25.555153 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:25.556475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:25.556763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:25.557042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] PATH_INFO: `/v3/users/7b192eacd6204f5c83ad63291dfcb04b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:25.558249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b192eacd6204f5c83ad63291dfcb04b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:25.560490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:25.609502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:25.611070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b192eacd6204f5c83ad63291dfcb04b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.619816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b192eacd6204f5c83ad63291dfcb04b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.632469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b192eacd6204f5c83ad63291dfcb04b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.633107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b192eacd6204f5c83ad63291dfcb04b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.640600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b192eacd6204f5c83ad63291dfcb04b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:25.647327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1005/2011] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:25 2022] DELETE /identity/v3/users/7b192eacd6204f5c83ad63291dfcb04b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:25.935650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:25.938127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:25.958792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d1682-136c-42df-95e8-998cf89f0e0a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.023092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.024620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.025075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.025364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] PATH_INFO: `/v3/projects/889914e46a7e4d6186d179de67fe9b22` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.026519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=889914e46a7e4d6186d179de67fe9b22)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.028478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.060549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.062156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.075328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eb6d7-18ff-4772-bc3d-3ce784001a4c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.091636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '889914e46a7e4d6186d179de67fe9b22', 'request_id': 'req-2c4c5686-20c2-4929-9777-1c4b0e796594'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.097333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '889914e46a7e4d6186d179de67fe9b22', 'request_id': 'req-2c4c5686-20c2-4929-9777-1c4b0e796594'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.104570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '889914e46a7e4d6186d179de67fe9b22', 'request_id': 'req-2c4c5686-20c2-4929-9777-1c4b0e796594'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.120435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] The token cache is being invalidate because project 889914e46a7e4d6186d179de67fe9b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:26.120868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.121905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2c4c5686-20c2-4929-9777-1c4b0e796594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.124889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1007/2012] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:25 2022] DELETE /identity/v3/projects/889914e46a7e4d6186d179de67fe9b22 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:26.129808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b079-addd-4b0a-93a7-b0cf82f5d02a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.130054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b079-addd-4b0a-93a7-b0cf82f5d02a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.130253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b079-addd-4b0a-93a7-b0cf82f5d02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.147736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.148530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.148936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.149399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] PATH_INFO: `/v3/users/45e8235b38184240ab7d7397d5603c15` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.150638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e8235b38184240ab7d7397d5603c15)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.151476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6628b079-addd-4b0a-93a7-b0cf82f5d02a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:26.152860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.153320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6628b079-addd-4b0a-93a7-b0cf82f5d02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.162806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1008/2013] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:26.170810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a8fb-6206-4ea5-9279-86f1c716614f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.171184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a8fb-6206-4ea5-9279-86f1c716614f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.171490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0a8fb-6206-4ea5-9279-86f1c716614f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.191211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-51d0a8fb-6206-4ea5-9279-86f1c716614f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:26.192544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-51d0a8fb-6206-4ea5-9279-86f1c716614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.200646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1009/2014] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:26.204891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0a8fb-6206-4ea5-9279-86f1c716614f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.213939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0a8fb-6206-4ea5-9279-86f1c716614f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.215390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e8235b38184240ab7d7397d5603c15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.220464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.221253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.221535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.221784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.222550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:26.222866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.224668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.225596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e8235b38184240ab7d7397d5603c15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.242275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1010/2015] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:03:26.242524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e8235b38184240ab7d7397d5603c15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.243557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.245006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e8235b38184240ab7d7397d5603c15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.255229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ccd50ac8-afc9-4c2c-a6ad-61da10811d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e8235b38184240ab7d7397d5603c15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.255822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da32695-24d7-4e27-81a9-1fb1a05ec69e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.261358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.262040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.262302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.262452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1006/2016] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:26 2022] DELETE /identity/v3/users/45e8235b38184240ab7d7397d5603c15 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:26.262599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.263356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.265331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.267038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.267329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.267591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.270901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:26.289975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:26.291588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.311212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1011/2017] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/projects => generated 393 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:26.312073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.313992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.336036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:26.381128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1007/2018] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:26.391617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.393148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.400120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac76a5-8ab3-4e0e-b72c-a624d13f69e5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.405900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.406502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.406898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.407271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.407888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.409918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.436436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1012/2019] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:26.436436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.437741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.451566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf3673-8524-4de9-aea1-a8cd0331d360 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.457055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.457897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.458222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.458443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] PATH_INFO: `/v3/users/bdc0e5ebef5b44dfb01105b0d3f08f80` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.459888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc0e5ebef5b44dfb01105b0d3f08f80)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.462032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.468486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ed1c3-3980-47d2-8f5b-08f7b96fe2d5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.516482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc0e5ebef5b44dfb01105b0d3f08f80'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.518201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc0e5ebef5b44dfb01105b0d3f08f80'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.528292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc0e5ebef5b44dfb01105b0d3f08f80'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.529289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdccb374-dacf-4a21-b26d-f59e8098fae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.530336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccb374-dacf-4a21-b26d-f59e8098fae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.530643 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccb374-dacf-4a21-b26d-f59e8098fae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.531080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdccb374-dacf-4a21-b26d-f59e8098fae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.532959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fdccb374-dacf-4a21-b26d-f59e8098fae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.549382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc0e5ebef5b44dfb01105b0d3f08f80'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.550270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc0e5ebef5b44dfb01105b0d3f08f80'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:26.567828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccb374-dacf-4a21-b26d-f59e8098fae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.567989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1013/2020] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:26 2022] DELETE /identity/v3/users/bdc0e5ebef5b44dfb01105b0d3f08f80 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:26.568837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.570037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdccb374-dacf-4a21-b26d-f59e8098fae8 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.570349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.582630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1008/2021] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:26.585147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016efa8a-806f-46d7-aa36-6ca925914adf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.585424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016efa8a-806f-46d7-aa36-6ca925914adf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.585694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016efa8a-806f-46d7-aa36-6ca925914adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.616301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-016efa8a-806f-46d7-aa36-6ca925914adf None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:26.617595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-016efa8a-806f-46d7-aa36-6ca925914adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.659042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:26.683871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed056b6a-cebd-4b93-9b11-40520d00146e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.689049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.689712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.689990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.690269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.690938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.693034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.699998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1009/2022] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:26.707141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1014/2023] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:26.708330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016efa8a-806f-46d7-aa36-6ca925914adf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.719670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.719996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016efa8a-806f-46d7-aa36-6ca925914adf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.721510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.724998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.725673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.725999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.726288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.727359 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.729178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.741382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1010/2024] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:26.748761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.760310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1852ae-9bf3-405b-a768-11d04a8d4f1b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.765159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6cc3e-cbef-4390-ad76-4c925aad6a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.765871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6cc3e-cbef-4390-ad76-4c925aad6a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.766108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6cc3e-cbef-4390-ad76-4c925aad6a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.766382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6cc3e-cbef-4390-ad76-4c925aad6a48 admin admin] PATH_INFO: `/v3/projects/c65852978d934a299c5c57dfce77697d/users/94bce9b4720d4c31892c9ae9fd62f2ac/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.785266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6cc3e-cbef-4390-ad76-4c925aad6a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65852978d934a299c5c57dfce77697d, user_id=94bce9b4720d4c31892c9ae9fd62f2ac, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.788094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b6cc3e-cbef-4390-ad76-4c925aad6a48 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.801147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272e019-b342-48f3-9f13-75b4601aebd5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:26.804780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1011/2025] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:26 2022] PUT /identity/v3/projects/c65852978d934a299c5c57dfce77697d/users/94bce9b4720d4c31892c9ae9fd62f2ac/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:26.816218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.816563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.816912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.831895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:26.850379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] MFA Rules not processed for user `94bce9b4720d4c31892c9ae9fd62f2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:26.851830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.878413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:26.879083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:26.879367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:26.879681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:26.945779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1012/2026] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:26.947066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:26.948420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:26.969430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:26.970948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:26.988431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1015/2027] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:26.989266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.001820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80b9c8-5fe5-4239-ad97-e096e10f457f service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.008248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.009242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.009800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.010346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.011041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.013451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.048403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774e8f6-59a3-4488-9df0-7f9842307ab9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.054210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.054922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.055245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.056034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] PATH_INFO: `/v3/projects/4afec509ee454beba68b0a8801edae64` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.057272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4afec509ee454beba68b0a8801edae64)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.059618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.062060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1016/2028] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:26 2022] POST /identity/v3/projects => generated 391 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:27.066129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.115853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:27.129649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4afec509ee454beba68b0a8801edae64', 'request_id': 'req-c6276a96-e668-4dc9-9968-71f79758e7b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.136588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4afec509ee454beba68b0a8801edae64', 'request_id': 'req-c6276a96-e668-4dc9-9968-71f79758e7b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.143779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4afec509ee454beba68b0a8801edae64', 'request_id': 'req-c6276a96-e668-4dc9-9968-71f79758e7b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.153304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07863820-2b19-4bac-9afd-61cfebea426f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.160001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] The token cache is being invalidate because project 4afec509ee454beba68b0a8801edae64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:27.161126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.162070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6276a96-e668-4dc9-9968-71f79758e7b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.164181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1013/2029] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:26 2022] DELETE /identity/v3/projects/4afec509ee454beba68b0a8801edae64 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:27.169443 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d167c-7e19-4559-820d-212e8319e65f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.169904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d167c-7e19-4559-820d-212e8319e65f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.170483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d167c-7e19-4559-820d-212e8319e65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.198159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-128d167c-7e19-4559-820d-212e8319e65f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:27.201587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-128d167c-7e19-4559-820d-212e8319e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.247559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:27.252730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.254105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.254583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.255015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.256961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.261457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.309663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1014/2030] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:27.309902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1017/2031] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:27 2022] POST /identity/v3/users => generated 302 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:27.313706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.314348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.314721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.315432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.317815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.342061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:27.343780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.350965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33b877-3431-4ea4-af2a-b3ef14a7769c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.356198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1015/2032] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:27.357690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.359321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.362485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.365060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.365450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.365778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.366755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.372271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.379254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37d526-9a28-4566-a260-d19965ca9594 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.387171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.389395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.392980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1018/2033] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:03:27.392980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.394786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.395356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] PATH_INFO: `/v3/projects/1c0a7236551541d3beaef2243709ed2c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.397101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c0a7236551541d3beaef2243709ed2c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.399961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.402412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd88b15-6add-47ce-a73f-86bb00aa8d7e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.414486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.415599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.416416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.416775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.417510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:27.418026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.419607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.436211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1019/2034] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:27.437200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.449936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39fcd7-b130-4f8f-bb39-2d67c37c5d63 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.462009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c0a7236551541d3beaef2243709ed2c', 'request_id': 'req-3ac689ad-fbb0-4ea6-818d-2d6951054c97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.469636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c0a7236551541d3beaef2243709ed2c', 'request_id': 'req-3ac689ad-fbb0-4ea6-818d-2d6951054c97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.476429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c0a7236551541d3beaef2243709ed2c', 'request_id': 'req-3ac689ad-fbb0-4ea6-818d-2d6951054c97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.496315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] The token cache is being invalidate because project 1c0a7236551541d3beaef2243709ed2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:27.496707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.497612 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.499706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1016/2035] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:27 2022] DELETE /identity/v3/projects/1c0a7236551541d3beaef2243709ed2c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:27.503759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.505469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.524471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.525416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.526034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.526680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] PATH_INFO: `/v3/projects/ea64e6ebfb4b44acb92da6da0b692fb8/users/6c2fc7f822cc4df4872d6b648097beae/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.546040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea64e6ebfb4b44acb92da6da0b692fb8, user_id=6c2fc7f822cc4df4872d6b648097beae, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.549053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.567490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1020/2036] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:27 2022] PUT /identity/v3/projects/ea64e6ebfb4b44acb92da6da0b692fb8/users/6c2fc7f822cc4df4872d6b648097beae/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:27.568566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.570601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.578014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac689ad-fbb0-4ea6-818d-2d6951054c97 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.597746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:27.634916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:27.647803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a5efa-1445-4809-b4c5-421e83353b78 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.649313 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.650186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.650532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.651018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.651670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.653675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.653876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.654609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.655046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.655454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] PATH_INFO: `/v3/users/0869261c946f47359209196f67f6dd78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.656769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=0869261c946f47359209196f67f6dd78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.658937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.687177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1017/2037] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:27 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:27.688029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.689700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.723089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0869261c946f47359209196f67f6dd78'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.724039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0869261c946f47359209196f67f6dd78'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.735894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0869261c946f47359209196f67f6dd78'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.751417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0869261c946f47359209196f67f6dd78'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.752253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0869261c946f47359209196f67f6dd78'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:27.762151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e1c70-e5d4-4b2b-a869-0cce63135340 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.766977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.767593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.767871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.768196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.771853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.771982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1021/2038] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:27 2022] DELETE /identity/v3/users/0869261c946f47359209196f67f6dd78 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:27.771982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.771982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.780222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.795651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1018/2039] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:27.801365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.803891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:27.818755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deecb87e-3722-48fb-90c5-1e0823ba75f3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.865524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7eb367-159b-4440-8e39-5c36033eb0ed admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.871594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.872288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.872565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.872835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.873495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.875491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.909429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1022/2040] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:27 2022] POST /identity/v3/users => generated 281 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:27.909604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff1408-4e78-4b91-af6a-7ab544ed7ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:27.909604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1408-4e78-4b91-af6a-7ab544ed7ead admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.909604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1408-4e78-4b91-af6a-7ab544ed7ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.909604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff1408-4e78-4b91-af6a-7ab544ed7ead admin admin] PATH_INFO: `/v3/projects/ea64e6ebfb4b44acb92da6da0b692fb8/users/6c2fc7f822cc4df4872d6b648097beae/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.910837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:27.911669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff1408-4e78-4b91-af6a-7ab544ed7ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea64e6ebfb4b44acb92da6da0b692fb8, user_id=6c2fc7f822cc4df4872d6b648097beae, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:27.914839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff1408-4e78-4b91-af6a-7ab544ed7ead admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:27.921609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0118fe-838b-4f13-85d6-bc3c4480faa2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:27.927115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1019/2041] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:27 2022] PUT /identity/v3/projects/ea64e6ebfb4b44acb92da6da0b692fb8/users/6c2fc7f822cc4df4872d6b648097beae/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:27.938539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:27.939063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:27.939594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:27.967313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:27.975530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] MFA Rules not processed for user `6c2fc7f822cc4df4872d6b648097beae`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:27.976824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.001537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.002193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.002467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.002739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.003442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.005112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.018852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1023/2042] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:28.019901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.030946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f27ebc-c06b-4ee2-8dd5-a8ad68a1a52d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.036253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:28.036726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.037347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.037628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.037920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.038869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.041277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.056216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1024/2043] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:28.057380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.064720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1020/2044] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:27 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:28.078017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.079673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.083641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0848a7-3586-4568-961a-1c1dd2e852ea admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.090324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.091233 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.091471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.091718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] PATH_INFO: `/v3/projects/fc7459c2f0804db9a696eee5dfb150a7/users/6f9a7ea06add44a2a87239a88641d3c5/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.120401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc7459c2f0804db9a696eee5dfb150a7, user_id=6f9a7ea06add44a2a87239a88641d3c5, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.124356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.145676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1025/2045] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:28 2022] PUT /identity/v3/projects/fc7459c2f0804db9a696eee5dfb150a7/users/6f9a7ea06add44a2a87239a88641d3c5/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:28.154911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be976bb-6e0f-42da-bc93-32de5f24973d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.156966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.159027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.199522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.200057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.200364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.200818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.229629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c2237-1834-4625-9208-585cf00063b2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.234508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.235165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.235466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.235743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] PATH_INFO: `/v3/users/8594238cf23746a8a5a45b7473e2f883` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.236888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8594238cf23746a8a5a45b7473e2f883)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.238810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.268766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.270343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.280781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1021/2046] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:28.285056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.286528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.288416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8594238cf23746a8a5a45b7473e2f883'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.289150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8594238cf23746a8a5a45b7473e2f883'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.298387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8594238cf23746a8a5a45b7473e2f883'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.301227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e160b3-3832-4ba6-ba9f-9864f41619c3 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.306919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.307561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.307843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.308123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] PATH_INFO: `/v3/projects/f4d56bdbb2d147a5b71ea2c6863e7b9f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.309582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d56bdbb2d147a5b71ea2c6863e7b9f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.311650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.313741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8594238cf23746a8a5a45b7473e2f883'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.314823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d90944d7-c63e-4270-ad28-313a72518aa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8594238cf23746a8a5a45b7473e2f883'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.329897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1026/2047] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:28 2022] DELETE /identity/v3/users/8594238cf23746a8a5a45b7473e2f883 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:28.331635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.331924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.332221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.354563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:28.363110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] MFA Rules not processed for user `6f9a7ea06add44a2a87239a88641d3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:28.365273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.365755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d56bdbb2d147a5b71ea2c6863e7b9f', 'request_id': 'req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.367939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:28.376824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d56bdbb2d147a5b71ea2c6863e7b9f', 'request_id': 'req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.384355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d56bdbb2d147a5b71ea2c6863e7b9f', 'request_id': 'req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.398324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] The token cache is being invalidate because project f4d56bdbb2d147a5b71ea2c6863e7b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:28.398618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.399439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df0b62cc-2c19-4b78-a6d0-c16fd654f96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.401025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1022/2048] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:28 2022] DELETE /identity/v3/projects/f4d56bdbb2d147a5b71ea2c6863e7b9f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:28.405517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6fa41-f530-490a-a030-04e7aa990fd6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.405799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6fa41-f530-490a-a030-04e7aa990fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.406071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6fa41-f530-490a-a030-04e7aa990fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.427061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c9c6fa41-f530-490a-a030-04e7aa990fd6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:28.428432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c9c6fa41-f530-490a-a030-04e7aa990fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.434329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1027/2049] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:28.435244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.436550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.496037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a54f-6115-43d4-833e-2bd94e59e518 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.499594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf792d3-9e82-4420-9950-1eae53fe742e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.500220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1023/2050] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:28.500350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf792d3-9e82-4420-9950-1eae53fe742e service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.500496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf792d3-9e82-4420-9950-1eae53fe742e service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.500770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf792d3-9e82-4420-9950-1eae53fe742e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.501557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.501928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.502119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.502662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dcf792d3-9e82-4420-9950-1eae53fe742e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.521872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:28.523254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.532042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1024/2051] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:28.533059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.535223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.547951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eec291-9617-4f31-9f19-893f59a89009 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.552249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.552984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.553261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.553607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.562775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.564213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf792d3-9e82-4420-9950-1eae53fe742e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.564349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.565650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf792d3-9e82-4420-9950-1eae53fe742e service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.573640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1025/2052] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:28.574676 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.575685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1028/2053] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:28.577080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.577373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.577649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.590870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.606713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:28.608203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.619134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1029/2054] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:28.620134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.621852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.634923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83663f-1e82-46d9-9523-b201caa3d63e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.643563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.644273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.644559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.644826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] PATH_INFO: `/v3/users/6f9a7ea06add44a2a87239a88641d3c5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.646542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9a7ea06add44a2a87239a88641d3c5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.649326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.649465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4c6eb-e76f-4b48-806a-590ad0a4e298 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.652656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.653883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.653883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.653883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.667341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.669243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.683843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1026/2055] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:28.686720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.707604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9d4a6-45c5-42fb-a4fe-2674571605fb service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.719381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.725797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.726092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.726372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.728047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:28.729127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.730385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.742603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:28.743793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9a7ea06add44a2a87239a88641d3c5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.745001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9a7ea06add44a2a87239a88641d3c5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.760008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1027/2056] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:28.768554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.770916 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9a7ea06add44a2a87239a88641d3c5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.771868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.790191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a5e2b-7641-4ea1-8fc9-7ca600057c9d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.796876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.798092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.798569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.799031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.800705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.801099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9a7ea06add44a2a87239a88641d3c5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.803085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c2ca6b65-428f-4594-83b0-d15362714a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9a7ea06add44a2a87239a88641d3c5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:28.803811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:28.824864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1030/2057] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:28 2022] DELETE /identity/v3/users/6f9a7ea06add44a2a87239a88641d3c5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:28.832354 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.832629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.832896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.848704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1028/2058] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:28 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:28.853290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.865819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:28.867169 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.906372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:28.959183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1031/2059] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:28.969251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed9627-b788-44bb-ab91-45b8e66d5c92 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:28.975016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:28.977945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:28.978409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:28.979154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:28.979434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:28.979768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:28.980405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:28.982983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.022009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1029/2060] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:28 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:29.022983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.044906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e347a54-cc0d-4fbc-98ee-a03daddf421b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.050147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.051055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.051342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.051616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.056851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.058630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.070391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1030/2061] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:29.074849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.083005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b559912-4f04-4982-bcd7-d0c1b164226b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.085911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1996f6-b2bd-4898-9c21-43a3fe14bb3a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.087842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.088765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.088985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.089344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.094399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.096004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.097739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.098120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.100614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.102711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.111105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.112641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.125039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1032/2062] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:29.130353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1031/2063] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:29.134523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.168567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f8874-062c-4792-9219-fe75eac2781c service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.182379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.182679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.183229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.184312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.185778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.186401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] PATH_INFO: `/v3/projects/b73ef0658c274f56bed5e705166d51f6/users/9b37ad685cfa45899675da40cbf28a2a/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.198549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9069ad-4bf6-4f60-ad25-01b34ed19c18 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.214076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73ef0658c274f56bed5e705166d51f6, user_id=9b37ad685cfa45899675da40cbf28a2a, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.218518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.218857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.220022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.220828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.221384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] PATH_INFO: `/v3/users/c73df39bdc524ea2a914cc8c732e6839` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.223585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73df39bdc524ea2a914cc8c732e6839)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.225965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.237987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1033/2064] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:29 2022] PUT /identity/v3/projects/b73ef0658c274f56bed5e705166d51f6/users/9b37ad685cfa45899675da40cbf28a2a/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:29.238886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.240572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.297200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73df39bdc524ea2a914cc8c732e6839'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.303526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:29.311776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73df39bdc524ea2a914cc8c732e6839'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.332101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73df39bdc524ea2a914cc8c732e6839'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.333304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73df39bdc524ea2a914cc8c732e6839'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.340876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3e4dc-3b3a-48cc-8484-bee5654caaf6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.341965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-52d0d976-aafb-4342-ae1f-15376c74f3f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73df39bdc524ea2a914cc8c732e6839'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.346402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.347148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.347433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.347999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] PATH_INFO: `/v3/projects/c3d7200adf6e4a3a8d4d220c187f29e8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.348423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1032/2065] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:29 2022] DELETE /identity/v3/users/c73df39bdc524ea2a914cc8c732e6839 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:29.349724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.350003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.350270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.352974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d7200adf6e4a3a8d4d220c187f29e8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.355900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.389229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] MFA Rules not processed for user `9b37ad685cfa45899675da40cbf28a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:29.391125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.421002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d7200adf6e4a3a8d4d220c187f29e8', 'request_id': 'req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.428058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d7200adf6e4a3a8d4d220c187f29e8', 'request_id': 'req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.436408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d7200adf6e4a3a8d4d220c187f29e8', 'request_id': 'req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.455781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] The token cache is being invalidate because project c3d7200adf6e4a3a8d4d220c187f29e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:29.456073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.456949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4000adad-66e4-47ea-9096-e7e5f1dc2eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.461273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1034/2066] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:29 2022] DELETE /identity/v3/projects/c3d7200adf6e4a3a8d4d220c187f29e8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:29.461273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.461273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.461273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.465675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:29.477449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1033/2067] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:29.483595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.486039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.488418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:29.489667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.549988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1035/2068] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:29.550848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.552112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.557162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d9d9d-61bd-4160-913f-dabdcb566196 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.562009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.562649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.562926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.563199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] PATH_INFO: `/v3/projects/fc7459c2f0804db9a696eee5dfb150a7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.564340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc7459c2f0804db9a696eee5dfb150a7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.564602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f9e5b-767c-451c-9a22-23b2ac6d9d42 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.566283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.569357 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.569983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.570260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.570552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] PATH_INFO: `/v3/users/d364eb61f11946669e485cedffbde299` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.571753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d364eb61f11946669e485cedffbde299)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.573707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.616783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc7459c2f0804db9a696eee5dfb150a7', 'request_id': 'req-665882cd-af48-42a8-abfc-d78a8b30678e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.623125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc7459c2f0804db9a696eee5dfb150a7', 'request_id': 'req-665882cd-af48-42a8-abfc-d78a8b30678e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.623572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd364eb61f11946669e485cedffbde299'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.624300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd364eb61f11946669e485cedffbde299'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.629149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc7459c2f0804db9a696eee5dfb150a7', 'request_id': 'req-665882cd-af48-42a8-abfc-d78a8b30678e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.633585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd364eb61f11946669e485cedffbde299'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.642335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:29.642547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] The token cache is being invalidate because project fc7459c2f0804db9a696eee5dfb150a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:29.643138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.643877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.645544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1034/2069] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:29 2022] DELETE /identity/v3/projects/fc7459c2f0804db9a696eee5dfb150a7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:29.646273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.647563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.651898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd364eb61f11946669e485cedffbde299'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.653227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd364eb61f11946669e485cedffbde299'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:29.667231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1036/2070] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:29 2022] DELETE /identity/v3/users/d364eb61f11946669e485cedffbde299 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:29.668418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.669744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.718338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665882cd-af48-42a8-abfc-d78a8b30678e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.736226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b03ac-1af4-4db7-b59e-819f6040a3aa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.740122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.740763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.741041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.741315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.743163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.760679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:29.795246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8051be-9525-4041-ae63-9d8d4413bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.796079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8051be-9525-4041-ae63-9d8d4413bf76 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.796224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8051be-9525-4041-ae63-9d8d4413bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.796489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8051be-9525-4041-ae63-9d8d4413bf76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.797139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8051be-9525-4041-ae63-9d8d4413bf76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.799074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8051be-9525-4041-ae63-9d8d4413bf76 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.849451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.851200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.857120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1035/2071] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:29 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:29.857120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c1e69-850b-498d-bd3b-01b31b202f9f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.857120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c1e69-850b-498d-bd3b-01b31b202f9f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.857120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c1e69-850b-498d-bd3b-01b31b202f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.865824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1037/2072] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:29.866567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:29.886174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-324c1e69-850b-498d-bd3b-01b31b202f9f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:29.887486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-324c1e69-850b-498d-bd3b-01b31b202f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:29.892970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fd0a3-d743-4478-8073-50031028f008 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:29.972415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:29.973163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.973471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.973738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:29.974622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:29.975786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1036/2073] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:29.977812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:29.984422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:29.984973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:29.986282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.003446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1038/2074] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:29 2022] POST /identity/v3/users => generated 298 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:30.011607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.022405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5987dd-40da-45a9-a6bd-eaf0ca499215 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.037358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:30.038889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.060431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.061078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.064226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.064684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1037/2075] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:30.064684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.065027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.066201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.066848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.068570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.080429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd0f47-76c4-4c87-a64f-7de157cfb99d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.082927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1039/2076] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:30.085401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.088466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.089150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.089430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.089704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] PATH_INFO: `/v3/projects/b322ec54abc148ab8ef427cbbcaddfb3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.090858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b322ec54abc148ab8ef427cbbcaddfb3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.093123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.098230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af9c3f-17ae-434a-afc6-8b46afa5fe7f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.104833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.105735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.106039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.106313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.107138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:30.107440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.109880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.118054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:30.128251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1040/2077] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:30.129857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.150001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677a3a1-6837-436d-a633-19039f20b229 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.171157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b322ec54abc148ab8ef427cbbcaddfb3', 'request_id': 'req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.180352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b322ec54abc148ab8ef427cbbcaddfb3', 'request_id': 'req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.188058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b322ec54abc148ab8ef427cbbcaddfb3', 'request_id': 'req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.208762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] The token cache is being invalidate because project b322ec54abc148ab8ef427cbbcaddfb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:30.208967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.210214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.213169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1038/2078] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:30 2022] DELETE /identity/v3/projects/b322ec54abc148ab8ef427cbbcaddfb3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:30.214370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.217847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.237000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156394ba-4345-4173-a9d2-825b704f43e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.237759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156394ba-4345-4173-a9d2-825b704f43e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.238245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156394ba-4345-4173-a9d2-825b704f43e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.238596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156394ba-4345-4173-a9d2-825b704f43e5 admin admin] PATH_INFO: `/v3/projects/e2ce4eaabb274bed874055fa8f34fca0/users/c304003d63cc48d890e992d038e381ff/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.261866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156394ba-4345-4173-a9d2-825b704f43e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ce4eaabb274bed874055fa8f34fca0, user_id=c304003d63cc48d890e992d038e381ff, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.265037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156394ba-4345-4173-a9d2-825b704f43e5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.277269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:30.293737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1041/2079] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:30 2022] PUT /identity/v3/projects/e2ce4eaabb274bed874055fa8f34fca0/users/c304003d63cc48d890e992d038e381ff/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:30.296694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.296694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.296694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.329615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13828ec-8eb8-47e6-b1b4-4d2b4c440660 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.334670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:30.336757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.420542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.421472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.421920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.422319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.423134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.425206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.435414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1042/2080] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:30.442436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.444341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.458250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc389b7b-0917-4573-950b-d73056e9fc2d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.464059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1039/2081] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:30.464059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.466021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.530424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.531076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.531353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.531624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] PATH_INFO: `/v3/projects/1ff96d76ce9e432fa63aa8a2feb80ec9` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.532767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ff96d76ce9e432fa63aa8a2feb80ec9)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.534699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.536961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb2e5a-c7fd-4a75-b4df-43c77b6e5978 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.542416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc1b278-3333-4da3-9a84-326af6c61907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.543366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1b278-3333-4da3-9a84-326af6c61907 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.543366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1b278-3333-4da3-9a84-326af6c61907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.543758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc1b278-3333-4da3-9a84-326af6c61907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.544475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1b278-3333-4da3-9a84-326af6c61907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.546269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc1b278-3333-4da3-9a84-326af6c61907 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.560043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1040/2082] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:30.560043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.560043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.560043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.566052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:30.582724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:30.583984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.594878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ff96d76ce9e432fa63aa8a2feb80ec9', 'request_id': 'req-5beb7d7a-8948-4819-9efa-4e347cad2af6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.601299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ff96d76ce9e432fa63aa8a2feb80ec9', 'request_id': 'req-5beb7d7a-8948-4819-9efa-4e347cad2af6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.608433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ff96d76ce9e432fa63aa8a2feb80ec9', 'request_id': 'req-5beb7d7a-8948-4819-9efa-4e347cad2af6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.624658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] The token cache is being invalidate because project 1ff96d76ce9e432fa63aa8a2feb80ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:30.625568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.625820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:30.627432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1043/2083] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:30 2022] DELETE /identity/v3/projects/1ff96d76ce9e432fa63aa8a2feb80ec9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:30.628269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.629565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.662104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1041/2084] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:30.666104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.666104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.677302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbbd2e-d917-4b43-aded-9298d22d6f5a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.682167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0c823-74ea-4b9a-8f48-250283ccda72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.682841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0c823-74ea-4b9a-8f48-250283ccda72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.683098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0c823-74ea-4b9a-8f48-250283ccda72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.683551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0c823-74ea-4b9a-8f48-250283ccda72 admin admin] PATH_INFO: `/v3/projects/e2ce4eaabb274bed874055fa8f34fca0/users/c304003d63cc48d890e992d038e381ff/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.685419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe0c823-74ea-4b9a-8f48-250283ccda72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ce4eaabb274bed874055fa8f34fca0, user_id=c304003d63cc48d890e992d038e381ff, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.688125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe0c823-74ea-4b9a-8f48-250283ccda72 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.690882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:30.701607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1042/2085] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:30 2022] PUT /identity/v3/projects/e2ce4eaabb274bed874055fa8f34fca0/users/c304003d63cc48d890e992d038e381ff/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:30.702966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53863d12-87db-498b-a2cc-ddbe40fe668d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.703659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53863d12-87db-498b-a2cc-ddbe40fe668d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.703932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53863d12-87db-498b-a2cc-ddbe40fe668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.716965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb7d7a-8948-4819-9efa-4e347cad2af6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.724844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-53863d12-87db-498b-a2cc-ddbe40fe668d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:30.726058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-53863d12-87db-498b-a2cc-ddbe40fe668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.782349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067e145-8df9-4441-bc96-1e338902387d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.783372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067e145-8df9-4441-bc96-1e338902387d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.783652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067e145-8df9-4441-bc96-1e338902387d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.783919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067e145-8df9-4441-bc96-1e338902387d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.784620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067e145-8df9-4441-bc96-1e338902387d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.786545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067e145-8df9-4441-bc96-1e338902387d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.797603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1043/2086] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:30.804043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53863d12-87db-498b-a2cc-ddbe40fe668d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.814346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53863d12-87db-498b-a2cc-ddbe40fe668d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.821001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1044/2087] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:30.821367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7107c4-9983-43f8-8a6d-7e8f00a0daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.822046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7107c4-9983-43f8-8a6d-7e8f00a0daa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.822364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7107c4-9983-43f8-8a6d-7e8f00a0daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.822756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b61a7a-1156-4a87-b779-64ccb17174e3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.822893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7107c4-9983-43f8-8a6d-7e8f00a0daa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.822972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b61a7a-1156-4a87-b779-64ccb17174e3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.823204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b61a7a-1156-4a87-b779-64ccb17174e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.823475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7107c4-9983-43f8-8a6d-7e8f00a0daa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:30.823893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7107c4-9983-43f8-8a6d-7e8f00a0daa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.825734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7107c4-9983-43f8-8a6d-7e8f00a0daa1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.844382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1044/2088] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:30.844382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b10e-6e37-4ca6-a466-b6edaaedd727 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.844382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b10e-6e37-4ca6-a466-b6edaaedd727 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.844382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f7b10e-6e37-4ca6-a466-b6edaaedd727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.860292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-72b61a7a-1156-4a87-b779-64ccb17174e3 None None] MFA Rules not processed for user `c304003d63cc48d890e992d038e381ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:30.861595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72b61a7a-1156-4a87-b779-64ccb17174e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.868633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c3f7b10e-6e37-4ca6-a466-b6edaaedd727 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:30.870555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c3f7b10e-6e37-4ca6-a466-b6edaaedd727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:30.881547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1045/2089] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:30.885458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7b10e-6e37-4ca6-a466-b6edaaedd727 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.898322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f7b10e-6e37-4ca6-a466-b6edaaedd727 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.905832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.906461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.906742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.907019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.907657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.909307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.913515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:30.924397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1046/2090] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:30.925433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.945131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21d256-9bea-4907-a229-74374dbc332f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:30.950132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:30.950865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:30.951231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:30.951546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:30.952296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:30.954472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:30.971498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1045/2091] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:30.975978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b61a7a-1156-4a87-b779-64ccb17174e3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:30.995622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1047/2092] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:30 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:30.996909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.003826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b61a7a-1156-4a87-b779-64ccb17174e3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.082422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.083625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.084022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.084343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.085492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:31.086138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.088566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.091788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b7964-c70e-40a3-b621-60f890390fe0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.098878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.099602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.099946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.100319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.101659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.103944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.114926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1046/2093] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:31.116397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.118159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1048/2094] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:31.119192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.121178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:31.129394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c17b5-33de-492e-9fc8-d3495820156e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.134628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.135436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.135980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.136481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.137296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.139384 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.169605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1047/2095] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:31 2022] POST /identity/v3/users => generated 292 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:31.173946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.174266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:31.194838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ee4a1-b90a-4664-ab8a-5dce7c963840 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.198684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f3833-807e-4287-8058-0213858f9d47 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.200338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.201060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.201360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.201654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] PATH_INFO: `/v3/projects/98950bc43f1c4b5a8b40812dea01ad8b/users/8f568f07546f49329b322adfc21684ca/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.204642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.207159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.207620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.208055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.223392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] RBAC: Authorizing `identity:create_grant(project_id=98950bc43f1c4b5a8b40812dea01ad8b, user_id=8f568f07546f49329b322adfc21684ca, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.226536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.249097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1048/2096] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:31 2022] PUT /identity/v3/projects/98950bc43f1c4b5a8b40812dea01ad8b/users/8f568f07546f49329b322adfc21684ca/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:31.250020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.302444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.303906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.314391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1049/2097] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:31.314947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.325702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2701a-654e-4e13-9e3d-3a29e90a9800 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.339201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb508e29-a8e2-43e2-a281-bab2b386328d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.344565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842d5d2-3b09-4b09-a626-fe273db2ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.345251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842d5d2-3b09-4b09-a626-fe273db2ec53 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.345534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842d5d2-3b09-4b09-a626-fe273db2ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.346228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842d5d2-3b09-4b09-a626-fe273db2ec53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.346904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842d5d2-3b09-4b09-a626-fe273db2ec53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.349812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842d5d2-3b09-4b09-a626-fe273db2ec53 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.386847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1049/2098] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:31 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:31.388365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98c4ee-b2dd-4e39-8297-3954209d66cd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.388648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98c4ee-b2dd-4e39-8297-3954209d66cd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.388947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98c4ee-b2dd-4e39-8297-3954209d66cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.393479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.394107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.394391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.394666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.395311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.396983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.410731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1050/2099] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:31.411621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.422620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf74aaa-3d58-4120-ab9b-a70c0124992c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.423432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:31.435291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5d98c4ee-b2dd-4e39-8297-3954209d66cd None None] MFA Rules not processed for user `8f568f07546f49329b322adfc21684ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:31.437241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5d98c4ee-b2dd-4e39-8297-3954209d66cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:31.484788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.485532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.485812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.486076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.486742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.488778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.506403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1050/2100] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:31.507259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98c4ee-b2dd-4e39-8297-3954209d66cd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.517523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1051/2101] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:31 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:31.517523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.522575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98c4ee-b2dd-4e39-8297-3954209d66cd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.528337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.528982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.529264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.529526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.530163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.531703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:31.531937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.547232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1051/2102] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:31.547232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.547232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:31.581486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a65706-a42b-4b4a-8479-96dbc44d20d9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.585345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.585962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.586241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.586566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.588395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:31.604276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c92044-d79c-45be-a569-e2ef5ee041d8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.607879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.608534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.608811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.609082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.618129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.619594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.628713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1052/2103] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:31.629514 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.640379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1584ca9-6bc2-4e11-a204-84abef4dc351 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.646066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.646671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.646951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.647230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.647876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.649805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.653017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.654618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.660879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1053/2104] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:31.661428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.666735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1052/2105] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:31.667569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.671850 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164d883-3f4d-41e4-92e9-af70e39cacfa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.677824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9700-0918-428f-8bfb-a194cecb74c7 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.680442 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f6984-3306-4c46-bb21-2a92e88b4fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.681335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f6984-3306-4c46-bb21-2a92e88b4fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.682734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f6984-3306-4c46-bb21-2a92e88b4fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.682866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.683322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f6984-3306-4c46-bb21-2a92e88b4fa9 admin admin] PATH_INFO: `/v3/projects/6e09929bc1514374977d20ecb3078851/users/bdb022dbd53546e5ac3cdc05335cf61b/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.683431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.683687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.683961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.684647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.686369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.698192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1053/2106] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:31.702646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.710485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f6984-3306-4c46-bb21-2a92e88b4fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e09929bc1514374977d20ecb3078851, user_id=bdb022dbd53546e5ac3cdc05335cf61b, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.713774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f6984-3306-4c46-bb21-2a92e88b4fa9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.713905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0e2eb-fc14-45fa-9ba3-47c0db34ee44 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:31.722064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6585f-a371-4e7c-b874-32ec670b4bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:31.722779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6585f-a371-4e7c-b874-32ec670b4bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.723111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6585f-a371-4e7c-b874-32ec670b4bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.723395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6585f-a371-4e7c-b874-32ec670b4bfa admin admin] PATH_INFO: `/v3/projects/7fc3899e9fd44ac08788f6191184bd3b/users/df785a13785b486e910d4f3a92e63140/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.736244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1054/2107] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:31 2022] PUT /identity/v3/projects/6e09929bc1514374977d20ecb3078851/users/bdb022dbd53546e5ac3cdc05335cf61b/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:31.745182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6585f-a371-4e7c-b874-32ec670b4bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc3899e9fd44ac08788f6191184bd3b, user_id=df785a13785b486e910d4f3a92e63140, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:31.747077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d8cf5-92ac-4963-bef8-31d61ddd5d97 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.748072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6585f-a371-4e7c-b874-32ec670b4bfa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:31.748556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d8cf5-92ac-4963-bef8-31d61ddd5d97 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.749771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d8cf5-92ac-4963-bef8-31d61ddd5d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.764611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1054/2108] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:31 2022] PUT /identity/v3/projects/7fc3899e9fd44ac08788f6191184bd3b/users/df785a13785b486e910d4f3a92e63140/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:31.775327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6a6dc-a050-453d-b8e5-a07d8174027a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:31.775842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6a6dc-a050-453d-b8e5-a07d8174027a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:31.776310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6a6dc-a050-453d-b8e5-a07d8174027a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:31.787419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5a6d8cf5-92ac-4963-bef8-31d61ddd5d97 None None] MFA Rules not processed for user `bdb022dbd53546e5ac3cdc05335cf61b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:31.788859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5a6d8cf5-92ac-4963-bef8-31d61ddd5d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:31.823355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6bb6a6dc-a050-453d-b8e5-a07d8174027a None None] MFA Rules not processed for user `df785a13785b486e910d4f3a92e63140`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:31.826062 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6bb6a6dc-a050-453d-b8e5-a07d8174027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:31.844428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:31.877588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1055/2109] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:31.909598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1055/2110] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:31.967613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d8cf5-92ac-4963-bef8-31d61ddd5d97 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:31.996337 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6a6dc-a050-453d-b8e5-a07d8174027a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:32.028963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:32.032024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d8cf5-92ac-4963-bef8-31d61ddd5d97 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:32.036436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:32.037184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:32.037525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:32.037870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:32.048924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:32.050749 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:32.061617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1056/2111] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:32.069389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6a6dc-a050-453d-b8e5-a07d8174027a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:32.073739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbb2bf-7e5e-44ce-8587-8f5d6f4f9fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:32.074401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb2bf-7e5e-44ce-8587-8f5d6f4f9fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:32.074798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb2bf-7e5e-44ce-8587-8f5d6f4f9fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:32.075283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb2bf-7e5e-44ce-8587-8f5d6f4f9fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:32.085158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbb2bf-7e5e-44ce-8587-8f5d6f4f9fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:32.087290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbb2bf-7e5e-44ce-8587-8f5d6f4f9fe5 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:32.105828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1056/2112] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:32.761793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:32.763533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:32.844073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04438a69-ad85-48d0-b214-1bad9436d296 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:32.848764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:32.849570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:32.849926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:32.850231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] PATH_INFO: `/v3/users/94bce9b4720d4c31892c9ae9fd62f2ac` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:32.851581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94bce9b4720d4c31892c9ae9fd62f2ac)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:32.854899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:32.907669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94bce9b4720d4c31892c9ae9fd62f2ac'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:32.908430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94bce9b4720d4c31892c9ae9fd62f2ac'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:32.917742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94bce9b4720d4c31892c9ae9fd62f2ac'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:32.930175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94bce9b4720d4c31892c9ae9fd62f2ac'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:32.930954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94bce9b4720d4c31892c9ae9fd62f2ac'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:32.943120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1057/2113] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:32 2022] DELETE /identity/v3/users/94bce9b4720d4c31892c9ae9fd62f2ac => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:32.948034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:32.948588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:32.948687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:32.971677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:32.973047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:33.034103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1057/2114] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:33.044549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:33.046239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:33.058402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:33.116071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba810d-f80f-46fd-b817-3c1a1bdf6d65 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:33.120505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:33.121243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:33.121595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:33.121842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:33.132218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:33.133986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:33.143862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1058/2115] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:33.422036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:33.423623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:33.451150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0066e0a-bb34-419f-9b4b-1c2e7f96f39c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:33.456845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:33.457500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:33.457783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:33.458053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] PATH_INFO: `/v3/projects/c65852978d934a299c5c57dfce77697d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:33.459184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65852978d934a299c5c57dfce77697d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:33.461157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:33.518548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65852978d934a299c5c57dfce77697d', 'request_id': 'req-31826e63-41d7-47cd-9669-bff1125bbc0c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.527079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65852978d934a299c5c57dfce77697d', 'request_id': 'req-31826e63-41d7-47cd-9669-bff1125bbc0c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.533688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65852978d934a299c5c57dfce77697d', 'request_id': 'req-31826e63-41d7-47cd-9669-bff1125bbc0c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.560845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] The token cache is being invalidate because project c65852978d934a299c5c57dfce77697d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:33.561216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.561947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.563535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1058/2116] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:33 2022] DELETE /identity/v3/projects/c65852978d934a299c5c57dfce77697d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:33.568258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:33.581982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e395a99-4e05-4e14-b6a1-931c28dbc407 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:33.652622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:33.653181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:33.656298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:33.656798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] PATH_INFO: `/v3/users/6c2fc7f822cc4df4872d6b648097beae` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:33.667510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c2fc7f822cc4df4872d6b648097beae)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:33.669723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:33.746128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c2fc7f822cc4df4872d6b648097beae'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.746781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c2fc7f822cc4df4872d6b648097beae'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.755947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c2fc7f822cc4df4872d6b648097beae'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.768642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c2fc7f822cc4df4872d6b648097beae'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.769366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-158349af-7857-47ab-a6ec-5c3f0aa4df18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c2fc7f822cc4df4872d6b648097beae'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:33.782920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1059/2117] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:33 2022] DELETE /identity/v3/users/6c2fc7f822cc4df4872d6b648097beae => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:34.136268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:34.137931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:34.152083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31826e63-41d7-47cd-9669-bff1125bbc0c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:34.208434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:34.229865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:34.230691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.230787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.231052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] PATH_INFO: `/v3/projects/ea64e6ebfb4b44acb92da6da0b692fb8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.232291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea64e6ebfb4b44acb92da6da0b692fb8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:34.234224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:34.286230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea64e6ebfb4b44acb92da6da0b692fb8', 'request_id': 'req-bc337446-7b1d-41f5-ba0d-831af12298da'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:34.292822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea64e6ebfb4b44acb92da6da0b692fb8', 'request_id': 'req-bc337446-7b1d-41f5-ba0d-831af12298da'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:34.298450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea64e6ebfb4b44acb92da6da0b692fb8', 'request_id': 'req-bc337446-7b1d-41f5-ba0d-831af12298da'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:34.312249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] The token cache is being invalidate because project ea64e6ebfb4b44acb92da6da0b692fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:34.312624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:34.313310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bc337446-7b1d-41f5-ba0d-831af12298da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:34.314889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1059/2118] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:34 2022] DELETE /identity/v3/projects/ea64e6ebfb4b44acb92da6da0b692fb8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:34.321593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91380fd6-379d-4653-9559-cb7e19346ce7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.321878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91380fd6-379d-4653-9559-cb7e19346ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.322153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91380fd6-379d-4653-9559-cb7e19346ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.342796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-91380fd6-379d-4653-9559-cb7e19346ce7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:34.344020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-91380fd6-379d-4653-9559-cb7e19346ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:34.374802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:34.414018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1060/2119] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:34.423492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e500a2-3361-4ec3-909a-bee77aec7007 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.423785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e500a2-3361-4ec3-909a-bee77aec7007 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.424061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e500a2-3361-4ec3-909a-bee77aec7007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.446051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-03e500a2-3361-4ec3-909a-bee77aec7007 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:34.447474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-03e500a2-3361-4ec3-909a-bee77aec7007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:34.457882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1060/2120] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:34.462448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91380fd6-379d-4653-9559-cb7e19346ce7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:34.472691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91380fd6-379d-4653-9559-cb7e19346ce7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:34.479470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:34.480303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.480643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.480992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.481979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:34.482326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:34.484835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:34.503271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1061/2121] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:34.510085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e500a2-3361-4ec3-909a-bee77aec7007 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:34.519972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e500a2-3361-4ec3-909a-bee77aec7007 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:34.525860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:34.526468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.526788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.527080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.527687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:34.529658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:34.560646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1061/2122] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:34 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:34.564947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:34.576715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64839e2-de66-4589-9b47-834a46e81832 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:34.643277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:34.643971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.645897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.646180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.646839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:34.648938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:34.685130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1062/2123] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:34 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:34.689989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:34.706942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be010b-574b-4edc-8ce0-011ee0cd2c75 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:34.714348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:34.715108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.715395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.715806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.716519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:34.718325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:34.730445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1062/2124] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:03:34.734527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:34.755016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98950e2-c843-4787-a6c7-e4d9e9d202cf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:34.760957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e0f9c-66f7-443c-a470-6f36b8e39727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:34.761366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0f9c-66f7-443c-a470-6f36b8e39727 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.761716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0f9c-66f7-443c-a470-6f36b8e39727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.761996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e0f9c-66f7-443c-a470-6f36b8e39727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.762683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e0f9c-66f7-443c-a470-6f36b8e39727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:34.764407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e0f9c-66f7-443c-a470-6f36b8e39727 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:34.777705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1063/2125] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:34.782369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:34.794616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb2614-3db8-4941-8922-ddeecd9deba2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:34.801285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:34.802663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.802954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.803241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] PATH_INFO: `/v3/projects/eacd048ece2a49b1ad61688cb8876a85/users/16e13919bdba44fc97a85fba5cf9c90e/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.825120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacd048ece2a49b1ad61688cb8876a85, user_id=16e13919bdba44fc97a85fba5cf9c90e, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:34.830953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:34.859398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1063/2126] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:34 2022] PUT /identity/v3/projects/eacd048ece2a49b1ad61688cb8876a85/users/16e13919bdba44fc97a85fba5cf9c90e/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:34.871950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140235a-8987-4801-8932-600717d5f90a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:34.872544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140235a-8987-4801-8932-600717d5f90a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:34.873241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140235a-8987-4801-8932-600717d5f90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:34.912308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4140235a-8987-4801-8932-600717d5f90a None None] MFA Rules not processed for user `16e13919bdba44fc97a85fba5cf9c90e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:34.913702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4140235a-8987-4801-8932-600717d5f90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:34.994159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1064/2127] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:35.006329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:35.008361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:35.069961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e597d-52f4-4903-bbd0-c8a7e0a45213 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:35.074223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:35.075001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:35.075429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:35.075715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:35.085113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:35.086641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:35.095766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1064/2128] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:35.119790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140235a-8987-4801-8932-600717d5f90a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:35.151752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:35.187761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140235a-8987-4801-8932-600717d5f90a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:35.193907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:35.194681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:35.195092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:35.195411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] PATH_INFO: `/v3/users/16e13919bdba44fc97a85fba5cf9c90e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:35.197261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e13919bdba44fc97a85fba5cf9c90e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:35.199609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:35.259572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e13919bdba44fc97a85fba5cf9c90e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.260395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e13919bdba44fc97a85fba5cf9c90e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.269889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e13919bdba44fc97a85fba5cf9c90e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.285900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e13919bdba44fc97a85fba5cf9c90e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.286981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-304a7c28-9050-4718-9e01-c67aeeb357d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e13919bdba44fc97a85fba5cf9c90e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.291485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:35.293041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:35.304654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1065/2129] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:35 2022] DELETE /identity/v3/users/16e13919bdba44fc97a85fba5cf9c90e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:35.307982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4290f78-880b-48a1-a7b2-b24fefc95036 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:35.309998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:35.310289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:35.310560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:35.325138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:35.337917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:35.339239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:35.379810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:35.380624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:35.380974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:35.381316 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] PATH_INFO: `/v3/users/8f568f07546f49329b322adfc21684ca` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:35.382594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f568f07546f49329b322adfc21684ca)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:35.385495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:35.416639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1066/2130] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:35.426855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:35.428312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:35.447053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f568f07546f49329b322adfc21684ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.455498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f568f07546f49329b322adfc21684ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.469193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f568f07546f49329b322adfc21684ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.469869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f568f07546f49329b322adfc21684ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.476779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c970c59f-f370-4b7a-b9e6-cbc894d1a318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f568f07546f49329b322adfc21684ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:35.483492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1065/2131] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:35 2022] DELETE /identity/v3/users/8f568f07546f49329b322adfc21684ca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:35.485309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:35.488906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:35.489432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:35.489800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:35.492061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926550f-e682-4f0b-a069-12c0bce2a7d1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:35.510645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:35.511853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:35.543080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:35.543699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:35.543980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:35.544288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:35.546058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:35.572180 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1066/2132] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:35.581701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:35.583116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:35.596102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f9fad-81a5-4770-8710-97382ad605e5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:35.599787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:35.600601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:35.600809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:35.601080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:35.610694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:35.612143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:35.613060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:35.614506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:35.621794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1067/2133] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:35.624613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1067/2134] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:36.041721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.045212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.062674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac85d86-cc00-4024-978e-832dc37979dd service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.069771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.070503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.071300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.071649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] PATH_INFO: `/v3/projects/eacd048ece2a49b1ad61688cb8876a85` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.073103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacd048ece2a49b1ad61688cb8876a85)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.075178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.080969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.082457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.097717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2353c-59d6-4b39-996f-c8a90259ccb4 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.103829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.104526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.104898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.105062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] PATH_INFO: `/v3/projects/98950bc43f1c4b5a8b40812dea01ad8b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.106225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98950bc43f1c4b5a8b40812dea01ad8b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.108186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.112053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:36.157373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacd048ece2a49b1ad61688cb8876a85', 'request_id': 'req-30a55627-2ee4-4fb3-9082-a96bd48622ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.164462 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacd048ece2a49b1ad61688cb8876a85', 'request_id': 'req-30a55627-2ee4-4fb3-9082-a96bd48622ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.166186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98950bc43f1c4b5a8b40812dea01ad8b', 'request_id': 'req-6ad23afe-1218-4ff4-94a0-0728be885b12'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.170997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacd048ece2a49b1ad61688cb8876a85', 'request_id': 'req-30a55627-2ee4-4fb3-9082-a96bd48622ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.176301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98950bc43f1c4b5a8b40812dea01ad8b', 'request_id': 'req-6ad23afe-1218-4ff4-94a0-0728be885b12'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.183357 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98950bc43f1c4b5a8b40812dea01ad8b', 'request_id': 'req-6ad23afe-1218-4ff4-94a0-0728be885b12'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.201221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] The token cache is being invalidate because project eacd048ece2a49b1ad61688cb8876a85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:36.201520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.202175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-30a55627-2ee4-4fb3-9082-a96bd48622ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.202343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] The token cache is being invalidate because project 98950bc43f1c4b5a8b40812dea01ad8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:36.202536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.203163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ad23afe-1218-4ff4-94a0-0728be885b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:36.203776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1068/2135] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:36 2022] DELETE /identity/v3/projects/eacd048ece2a49b1ad61688cb8876a85 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:36.204715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1068/2136] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:36 2022] DELETE /identity/v3/projects/98950bc43f1c4b5a8b40812dea01ad8b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:36.208848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d545e-5c2d-44d9-8722-c977107a2430 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.209134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d545e-5c2d-44d9-8722-c977107a2430 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.209358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30066ebb-6c1e-4cb7-adc9-eb23771a5d70 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.209465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d545e-5c2d-44d9-8722-c977107a2430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.209788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30066ebb-6c1e-4cb7-adc9-eb23771a5d70 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.210113 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30066ebb-6c1e-4cb7-adc9-eb23771a5d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.230075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-30066ebb-6c1e-4cb7-adc9-eb23771a5d70 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:36.230292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b36d545e-5c2d-44d9-8722-c977107a2430 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:36.231389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-30066ebb-6c1e-4cb7-adc9-eb23771a5d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.231497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b36d545e-5c2d-44d9-8722-c977107a2430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.263445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:36.306348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1069/2137] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:36.314072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1069/2138] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:36.314466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205ad5e-6719-4a8d-afa7-71a936b32f51 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.314800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205ad5e-6719-4a8d-afa7-71a936b32f51 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.315127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205ad5e-6719-4a8d-afa7-71a936b32f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.322929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755d49d-f9e0-4feb-8f33-1880a343dfc5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.323212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755d49d-f9e0-4feb-8f33-1880a343dfc5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.323479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755d49d-f9e0-4feb-8f33-1880a343dfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.335389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4205ad5e-6719-4a8d-afa7-71a936b32f51 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:36.336913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4205ad5e-6719-4a8d-afa7-71a936b32f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.343054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9755d49d-f9e0-4feb-8f33-1880a343dfc5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:36.344302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9755d49d-f9e0-4feb-8f33-1880a343dfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.345353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1070/2139] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:36.349750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205ad5e-6719-4a8d-afa7-71a936b32f51 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.352792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1070/2140] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:36.356829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755d49d-f9e0-4feb-8f33-1880a343dfc5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.359811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205ad5e-6719-4a8d-afa7-71a936b32f51 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.365190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.365678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755d49d-f9e0-4feb-8f33-1880a343dfc5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.365811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.366063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.366330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.366997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:36.367290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.369334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.371106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.371776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.372118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.372504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.373212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:36.373572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.375195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.383858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1071/2141] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:36.388220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.393846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1071/2142] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:03:36.394816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.399449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6484025-a370-4ad0-a7d0-986f6dbd82d3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.404314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.404913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.405187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.405452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.405559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655684c-8e17-4f9c-bb09-0eeeedd5aee8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.406078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.407995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.410327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.410933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.411220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.411491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.412119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.414060 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.440595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1072/2143] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:36.444814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.445880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1072/2144] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:36.450083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.511032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f7f6b-0487-4961-8ee4-9f589c3569cc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.515761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.516330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.516610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.516885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.517515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.519485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.527419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea78c2-a1d5-420d-8952-ca7813342a87 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.532036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:36.535249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.535952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.536271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.536538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.537269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.540221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.550867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1073/2145] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:36.552394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.565005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb6e6d-ef1c-444d-bf50-7b599b9289df admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.570457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1073/2146] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:36.570673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.570788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.571347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.571466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.572125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.572277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.573774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.582279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cbfdb-8c85-47fc-9266-b4122340b52c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.588098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1074/2147] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:36.590193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.590843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.591121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.591397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.592067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.593769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.594229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.604959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1074/2148] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:36.609240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.612171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a89763-940f-4156-aa8c-1741608ed803 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.617062 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.618380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.618656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.618985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.619397 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f20d-3b57-421d-ab22-5abe7848a893 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.619623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.621272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.624342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.624929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.625205 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.625476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.626109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.627725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.636018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1075/2149] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:36.636933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.642625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1075/2150] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:03:36.643691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.646843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f117ea-81bf-4317-8d23-b83e2ce177d6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.651499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde009d7-2883-4238-b1d1-a2d938aa4b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.652193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde009d7-2883-4238-b1d1-a2d938aa4b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.652474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde009d7-2883-4238-b1d1-a2d938aa4b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.652746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde009d7-2883-4238-b1d1-a2d938aa4b99 admin admin] PATH_INFO: `/v3/projects/36f96724a7414080941d272a58a43005/users/7c4ff4d67a0846a79b870fe7b5962638/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.653545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334c606-fd81-476c-9130-b49c904c4e16 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.658138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d0ace-552e-4f91-a900-b3ca3f9a1e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.658768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0ace-552e-4f91-a900-b3ca3f9a1e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.659045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0ace-552e-4f91-a900-b3ca3f9a1e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.659313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d0ace-552e-4f91-a900-b3ca3f9a1e0e admin admin] PATH_INFO: `/v3/projects/a8ce63941ef94823a9471a4d9ef5cde4/users/99f8c8a010ef43e18fe2d5bd0930fbea/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.669048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde009d7-2883-4238-b1d1-a2d938aa4b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f96724a7414080941d272a58a43005, user_id=7c4ff4d67a0846a79b870fe7b5962638, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.671726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde009d7-2883-4238-b1d1-a2d938aa4b99 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.674235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d0ace-552e-4f91-a900-b3ca3f9a1e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ce63941ef94823a9471a4d9ef5cde4, user_id=99f8c8a010ef43e18fe2d5bd0930fbea, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.677118 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d0ace-552e-4f91-a900-b3ca3f9a1e0e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.686087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1076/2151] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:36 2022] PUT /identity/v3/projects/36f96724a7414080941d272a58a43005/users/7c4ff4d67a0846a79b870fe7b5962638/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:36.693078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1076/2152] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:36 2022] PUT /identity/v3/projects/a8ce63941ef94823a9471a4d9ef5cde4/users/99f8c8a010ef43e18fe2d5bd0930fbea/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:36.700333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bebe09-fab7-49c2-bc9d-2d5ce0e132d1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.700333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bebe09-fab7-49c2-bc9d-2d5ce0e132d1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.700333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bebe09-fab7-49c2-bc9d-2d5ce0e132d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.703520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686edb-4e04-4f8c-ba4a-eba6cd7ea0bb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.703800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686edb-4e04-4f8c-ba4a-eba6cd7ea0bb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.704071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09686edb-4e04-4f8c-ba4a-eba6cd7ea0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.725880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f4bebe09-fab7-49c2-bc9d-2d5ce0e132d1 None None] MFA Rules not processed for user `7c4ff4d67a0846a79b870fe7b5962638`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:36.727533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4bebe09-fab7-49c2-bc9d-2d5ce0e132d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.733334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-09686edb-4e04-4f8c-ba4a-eba6cd7ea0bb None None] MFA Rules not processed for user `99f8c8a010ef43e18fe2d5bd0930fbea`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:36.734571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-09686edb-4e04-4f8c-ba4a-eba6cd7ea0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:36.761753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:36.792274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1077/2153] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:36.797006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bebe09-fab7-49c2-bc9d-2d5ce0e132d1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.802941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1077/2154] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:36.807095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686edb-4e04-4f8c-ba4a-eba6cd7ea0bb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.860005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bebe09-fab7-49c2-bc9d-2d5ce0e132d1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.864572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.865204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.865482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.865752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.866391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.868532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.870976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09686edb-4e04-4f8c-ba4a-eba6cd7ea0bb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.876080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.876726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.877006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.877299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.878382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.880786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:36.901385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1078/2155] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:36.902392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.913602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1078/2156] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:36.915043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:36.931130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:36.968954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3929c5-6a31-4286-ada3-c8f016b30bb7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.980187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13097a5-4adf-4a89-a8d9-6624f2d6fd51 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:36.985279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:36.985893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:36.986195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:36.986465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:36.987209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:36.989111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.018687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1079/2157] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:37.023157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.043455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb53a9f-357f-489f-a02f-ff3ef3e7ceb1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.048114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.048291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.049019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.049411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.049555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.050136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.050266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.050662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.051073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.051812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.052105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.054785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.065572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1080/2158] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:37.066364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.077607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f9302-120b-4d73-9fc5-34cf95f0179d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.082861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1079/2159] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:36 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:37.083243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.083951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.084263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.084653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.085398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.087383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.089203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.099469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1081/2160] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:03:37.099643 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa4f5d-4fd5-4d77-89a7-cf530f8f6a04 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.103950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.104852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.105467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.105771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.106016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.106661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.108317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.114281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7ee2-2849-4c2a-93fb-c923c9430e1f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.118974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fea09-ab70-4a2a-af7e-4a36fe35f7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.119622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fea09-ab70-4a2a-af7e-4a36fe35f7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.120368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1080/2161] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:37.120498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fea09-ab70-4a2a-af7e-4a36fe35f7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.120498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fea09-ab70-4a2a-af7e-4a36fe35f7fd admin admin] PATH_INFO: `/v3/projects/ee5c4704d30f48a5b7800e6a31715b71/users/32caf2fa1d934f3a8aa4ca3a99342adf/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.123976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.133817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed9132-4bc2-473a-8f98-22bd928c7b5c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.135860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fea09-ab70-4a2a-af7e-4a36fe35f7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5c4704d30f48a5b7800e6a31715b71, user_id=32caf2fa1d934f3a8aa4ca3a99342adf, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.138506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.138795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fea09-ab70-4a2a-af7e-4a36fe35f7fd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.139155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.139431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.139700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] PATH_INFO: `/v3/projects/990ad881b6be41218782ed5c68bee97e/users/d9f8dcac1d0e4b84a07c4efecadf93b3/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.141900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:37.155273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1082/2162] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:37 2022] PUT /identity/v3/projects/ee5c4704d30f48a5b7800e6a31715b71/users/32caf2fa1d934f3a8aa4ca3a99342adf/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:37.157218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990ad881b6be41218782ed5c68bee97e, user_id=d9f8dcac1d0e4b84a07c4efecadf93b3, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.159896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.166046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad44f5-2d2d-4e7b-ae30-fee243331f04 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.166327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad44f5-2d2d-4e7b-ae30-fee243331f04 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.166602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad44f5-2d2d-4e7b-ae30-fee243331f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.175423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1081/2163] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:37 2022] PUT /identity/v3/projects/990ad881b6be41218782ed5c68bee97e/users/d9f8dcac1d0e4b84a07c4efecadf93b3/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:37.179426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.194460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4cad44f5-2d2d-4e7b-ae30-fee243331f04 None None] MFA Rules not processed for user `32caf2fa1d934f3a8aa4ca3a99342adf`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:37.195715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4cad44f5-2d2d-4e7b-ae30-fee243331f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:37.246155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913d9a1-7b1f-4fa7-8c6d-c68a6b28d643 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.251103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.251722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.252002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.252336 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.252987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.254618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.265345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1083/2164] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:37.270270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1082/2165] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:37.270457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad44f5-2d2d-4e7b-ae30-fee243331f04 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.270554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.281512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607b554-699e-424d-978b-243fd674e5b1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.283544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad44f5-2d2d-4e7b-ae30-fee243331f04 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.286197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b512b-0800-4fd5-b452-f0da9a3c68d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.286920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b512b-0800-4fd5-b452-f0da9a3c68d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.287154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b512b-0800-4fd5-b452-f0da9a3c68d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.287435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b512b-0800-4fd5-b452-f0da9a3c68d0 admin admin] PATH_INFO: `/v3/projects/990ad881b6be41218782ed5c68bee97e/users/d9f8dcac1d0e4b84a07c4efecadf93b3/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.288218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.288770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.289043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.289310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.289488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b512b-0800-4fd5-b452-f0da9a3c68d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990ad881b6be41218782ed5c68bee97e, user_id=d9f8dcac1d0e4b84a07c4efecadf93b3, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.290025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.292032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.292218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b512b-0800-4fd5-b452-f0da9a3c68d0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.303849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1083/2166] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:37 2022] PUT /identity/v3/projects/990ad881b6be41218782ed5c68bee97e/users/d9f8dcac1d0e4b84a07c4efecadf93b3/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:37.314240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.314854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.314854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.328309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1084/2167] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:37 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:37.332415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.341430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:37.351546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] MFA Rules not processed for user `d9f8dcac1d0e4b84a07c4efecadf93b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:37.354262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:37.414076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ae50e-8e9b-4cb0-81fa-8111fe6e4a63 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.419208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.419903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.420288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.420631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.421274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.423218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.426832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1084/2168] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:37.437901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.439327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:37.451756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1085/2169] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:37 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:37.452966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.465172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52ecef-328d-488f-9a09-d0235a78ddf3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.471969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.472692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.473035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.473662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.474319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.475957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.487068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1086/2170] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:37.491361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.501429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258cd6f-c00d-46b7-b64b-a415aab5373b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.502264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4f8b6-f6f6-4486-91e1-87cd78bc0f5f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.505917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.506109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.506529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.506751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.506850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.507032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.507135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.507306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] PATH_INFO: `/v3/projects/2e03421e2c1f43b599e6d7a03c0c6576/users/59fdd66ccd314edcb87d802e2273702b/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.522768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e03421e2c1f43b599e6d7a03c0c6576, user_id=59fdd66ccd314edcb87d802e2273702b, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.525976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.540597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1087/2171] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:37 2022] PUT /identity/v3/projects/2e03421e2c1f43b599e6d7a03c0c6576/users/59fdd66ccd314edcb87d802e2273702b/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:37.545139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.565050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:37.566941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.568441 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.578002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1085/2172] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:37.613428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c1af1-ded9-41a5-a2f9-85f32d25da08 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.621799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f4016-ca26-434e-b541-2800a6fb9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.623082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f4016-ca26-434e-b541-2800a6fb9193 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.623082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f4016-ca26-434e-b541-2800a6fb9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.623415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f4016-ca26-434e-b541-2800a6fb9193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.624192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f4016-ca26-434e-b541-2800a6fb9193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.626819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f4016-ca26-434e-b541-2800a6fb9193 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.637735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1088/2173] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:03:37.642317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.654594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c48c59-703c-4845-afed-a6ee370c4cca service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.659618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.660323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.660601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.660874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] PATH_INFO: `/v3/projects/2e03421e2c1f43b599e6d7a03c0c6576/users/59fdd66ccd314edcb87d802e2273702b/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.663510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e03421e2c1f43b599e6d7a03c0c6576, user_id=59fdd66ccd314edcb87d802e2273702b, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.668887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.677892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1086/2174] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:37 2022] PUT /identity/v3/projects/2e03421e2c1f43b599e6d7a03c0c6576/users/59fdd66ccd314edcb87d802e2273702b/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:37.689499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.689759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.690026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.721467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] MFA Rules not processed for user `59fdd66ccd314edcb87d802e2273702b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:37.723109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:37.818281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1089/2175] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:37.830411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:37.833133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:37.879489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:37.898669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4ca7d-319e-4460-ade2-db27b576fcf5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:37.902811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73139ff-b2b3-47e0-a0a3-41801bf75ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:37.903468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73139ff-b2b3-47e0-a0a3-41801bf75ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:37.903693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73139ff-b2b3-47e0-a0a3-41801bf75ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:37.903967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73139ff-b2b3-47e0-a0a3-41801bf75ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:37.971617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73139ff-b2b3-47e0-a0a3-41801bf75ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:37.973098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73139ff-b2b3-47e0-a0a3-41801bf75ac5 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:37.983002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1087/2176] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:41.294544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:41.296308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:41.417069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee2759-36be-4ce4-8498-f351f9732af2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:41.422192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:41.422924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:41.423276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:41.423694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] PATH_INFO: `/v3/users/9b37ad685cfa45899675da40cbf28a2a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:41.424835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b37ad685cfa45899675da40cbf28a2a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:41.426960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:41.536493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:41.546057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b37ad685cfa45899675da40cbf28a2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:41.547629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b37ad685cfa45899675da40cbf28a2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:41.559369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b37ad685cfa45899675da40cbf28a2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:41.579657 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b37ad685cfa45899675da40cbf28a2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:41.580914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b37ad685cfa45899675da40cbf28a2a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:41.602539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1090/2177] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:41 2022] DELETE /identity/v3/users/9b37ad685cfa45899675da40cbf28a2a => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:41.607273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:41.607755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:41.607833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:41.631175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:41.632601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:41.697793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1088/2178] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:41.709572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:41.711320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:41.717844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:41.720082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:41.782697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92559831-510b-41eb-a21f-ed254c7d3a22 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:41.785523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf6f37-5e12-40f8-8a52-87570c34369a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:41.787683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:41.789289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:41.789702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:41.790301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:41.790512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:41.791214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:41.791503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:41.791801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:41.794602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:41.802744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:41.804219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:41.816110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1091/2179] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:41.850858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:41.869739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:41.871761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:41.884289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1089/2180] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:42.137211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:42.138763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.161709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0541a1f-1f70-472e-b768-e36ced4e44ff service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:42.167489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:42.168290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:42.168381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:42.169483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] PATH_INFO: `/v3/projects/b73ef0658c274f56bed5e705166d51f6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:42.171071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73ef0658c274f56bed5e705166d51f6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:42.173435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:42.231680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73ef0658c274f56bed5e705166d51f6', 'request_id': 'req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.237410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73ef0658c274f56bed5e705166d51f6', 'request_id': 'req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.244520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73ef0658c274f56bed5e705166d51f6', 'request_id': 'req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.260225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] The token cache is being invalidate because project b73ef0658c274f56bed5e705166d51f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:42.260647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.261304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.262845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1092/2181] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:42 2022] DELETE /identity/v3/projects/b73ef0658c274f56bed5e705166d51f6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:42.267164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:42.282340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660f7e0-6b50-4437-b753-d81b9cc77d6b service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:42.355997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:42.358621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.358882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:42.359763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:42.360198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:42.360627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] PATH_INFO: `/v3/users/c304003d63cc48d890e992d038e381ff` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:42.361926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c304003d63cc48d890e992d038e381ff)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:42.364282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:42.380482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f18ae-bd51-449d-861d-1c3c4fd63dbe admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:42.396313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:42.399267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:42.399656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:42.400128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] PATH_INFO: `/v3/users/7c4ff4d67a0846a79b870fe7b5962638` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:42.404104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4ff4d67a0846a79b870fe7b5962638)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:42.406217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:42.463096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:42.468613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4ff4d67a0846a79b870fe7b5962638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.469343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4ff4d67a0846a79b870fe7b5962638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.475670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c304003d63cc48d890e992d038e381ff'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.482828 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4ff4d67a0846a79b870fe7b5962638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.491279 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c304003d63cc48d890e992d038e381ff'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.499942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4ff4d67a0846a79b870fe7b5962638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.509006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c304003d63cc48d890e992d038e381ff'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.509865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c304003d63cc48d890e992d038e381ff'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.512590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ce6144c3-b0df-44ee-9038-7a8436e66e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4ff4d67a0846a79b870fe7b5962638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.519205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c304003d63cc48d890e992d038e381ff'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.530007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1090/2182] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:42 2022] DELETE /identity/v3/users/c304003d63cc48d890e992d038e381ff => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:42.530729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:42.535548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1093/2183] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:42 2022] DELETE /identity/v3/users/7c4ff4d67a0846a79b870fe7b5962638 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:42.535713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.537148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:42.537509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:42.538075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:42.566070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:42.567640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.658705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1094/2184] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:42.659747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:42.661201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.670306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ad28c-10ed-4adc-b157-a2aa96c81906 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:42.683990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:42.684814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:42.685159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:42.685430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] PATH_INFO: `/v3/users/df785a13785b486e910d4f3a92e63140` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:42.687091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df785a13785b486e910d4f3a92e63140)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:42.689435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:42.695065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:42.733896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf00b53-3f90-47d3-9e82-b236ac4425f0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:42.739554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:42.740709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:42.741047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:42.741371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:42.743754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.788471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df785a13785b486e910d4f3a92e63140'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.804398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df785a13785b486e910d4f3a92e63140'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.823377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df785a13785b486e910d4f3a92e63140'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.824137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df785a13785b486e910d4f3a92e63140'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.835126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:42.836816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:42.841164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3754ffa9-634a-43c7-bf48-3d039da26658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df785a13785b486e910d4f3a92e63140'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:42.849267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1095/2185] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:42.853592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1091/2186] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:42 2022] DELETE /identity/v3/users/df785a13785b486e910d4f3a92e63140 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:42.858941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:42.859236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:42.859548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:42.861351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:42.864451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.881447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:42.882103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:42.883349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:42.969826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530907dd-7342-4c3f-bc3c-f2c465df94da service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:42.987170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1092/2187] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:42.988705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:42.990458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.023602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c6e2b-3c5c-4105-9d54-abf731cdbb30 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.028758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.029429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.029708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.030137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] PATH_INFO: `/v3/projects/e2ce4eaabb274bed874055fa8f34fca0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.031276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ce4eaabb274bed874055fa8f34fca0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.033260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.048304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.049004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.049363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.049727 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.051670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.064322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.065803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.075326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1096/2188] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:43.076112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.077575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.083586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ce4eaabb274bed874055fa8f34fca0', 'request_id': 'req-23e97398-eee4-4780-a978-dbe346937b91'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.089019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc06f70-0bd8-4ea6-aeec-bd354b1aa195 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.091216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ce4eaabb274bed874055fa8f34fca0', 'request_id': 'req-23e97398-eee4-4780-a978-dbe346937b91'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.098469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ce4eaabb274bed874055fa8f34fca0', 'request_id': 'req-23e97398-eee4-4780-a978-dbe346937b91'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.121663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] The token cache is being invalidate because project e2ce4eaabb274bed874055fa8f34fca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:43.122885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.123819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-23e97398-eee4-4780-a978-dbe346937b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.128175 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1093/2189] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:42 2022] DELETE /identity/v3/projects/e2ce4eaabb274bed874055fa8f34fca0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:43.133374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b14e8-b2fb-4086-a4fa-54850aa2a3f8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.133996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b14e8-b2fb-4086-a4fa-54850aa2a3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.134598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b14e8-b2fb-4086-a4fa-54850aa2a3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.155471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.156537 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:43.156710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.157946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.158903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.162036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.167507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d39b14e8-b2fb-4086-a4fa-54850aa2a3f8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:43.169374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d39b14e8-b2fb-4086-a4fa-54850aa2a3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.257960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.261352 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.276984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1094/2190] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:43.288829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f868d-3033-4474-8cea-a49c2a4df77b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.289419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f868d-3033-4474-8cea-a49c2a4df77b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.289699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f868d-3033-4474-8cea-a49c2a4df77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.291722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1097/2191] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:43.324558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.326659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.342058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c65f868d-3033-4474-8cea-a49c2a4df77b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:43.343555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c65f868d-3033-4474-8cea-a49c2a4df77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.349191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464c5cc-7d58-4f65-8516-2b60b4aeddc2 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.356552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.357438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.357871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.358147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] PATH_INFO: `/v3/users/bdb022dbd53546e5ac3cdc05335cf61b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.359446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdb022dbd53546e5ac3cdc05335cf61b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.361986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.371234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1095/2192] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:43.372504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f868d-3033-4474-8cea-a49c2a4df77b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.404283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:43.404433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f868d-3033-4474-8cea-a49c2a4df77b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.404433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.404433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.404433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.404736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.404736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:43.404736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.404736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.423677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1096/2193] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:43.425263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.437971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bb839-1531-4e48-80f4-30089bc835be admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.444584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.445273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.445552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.445825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.446468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.450594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.471030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdb022dbd53546e5ac3cdc05335cf61b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.472231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdb022dbd53546e5ac3cdc05335cf61b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.497382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdb022dbd53546e5ac3cdc05335cf61b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.526034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1097/2194] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:43 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:43.527825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdb022dbd53546e5ac3cdc05335cf61b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.527948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.529108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3ca97ae-2192-4473-b31d-8d7dab286be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdb022dbd53546e5ac3cdc05335cf61b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.547186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.556899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1098/2195] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:43 2022] DELETE /identity/v3/users/bdb022dbd53546e5ac3cdc05335cf61b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:43.558449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.559342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.559625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.613892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:43.615405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.656737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65c30e-e9d8-468d-9ef7-56d3e3158984 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.736567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1099/2196] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:43.737438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.742226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.774616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fedf6a-a318-4ae6-b939-0e867ad43ef2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.775688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.776365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.776648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.776924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.777568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.780345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.790000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:43.790717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:43.793691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:43.793970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] PATH_INFO: `/v3/projects/36f96724a7414080941d272a58a43005` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:43.795701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f96724a7414080941d272a58a43005)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:43.800831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:43.839675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1098/2197] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:43 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:43.840482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.848903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.861000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:43.907489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f96724a7414080941d272a58a43005', 'request_id': 'req-96ea9faf-f51b-4dd8-aebe-1edd592b105b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.917323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f96724a7414080941d272a58a43005', 'request_id': 'req-96ea9faf-f51b-4dd8-aebe-1edd592b105b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.927661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f96724a7414080941d272a58a43005', 'request_id': 'req-96ea9faf-f51b-4dd8-aebe-1edd592b105b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.934432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173cb94-3776-48d8-96af-35b6f106ef10 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:43.954741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] The token cache is being invalidate because project 36f96724a7414080941d272a58a43005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:43.955197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.958383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:43.960917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1100/2198] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:43 2022] DELETE /identity/v3/projects/36f96724a7414080941d272a58a43005 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:43.962005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:43.964399 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:43.989497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:44.021803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.022630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.022970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.023562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.026518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.074225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea9faf-f51b-4dd8-aebe-1edd592b105b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.081943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.082853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.083607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.083958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.084734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.087167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.108526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1101/2199] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:44.109630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.111316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.131113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.132721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.141600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adbbff-08cf-4a93-9741-091974fb3506 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.148682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.149902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.150263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.150536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] PATH_INFO: `/v3/projects/7fc3899e9fd44ac08788f6191184bd3b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.169252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc3899e9fd44ac08788f6191184bd3b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.171189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.179151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1099/2200] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:44.188901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.198059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.226724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790136c-d48b-4fa5-ae03-35d09f0e327c service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.237465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.238439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.239488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.239758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] PATH_INFO: `/v3/users/d9f8dcac1d0e4b84a07c4efecadf93b3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.241614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f8dcac1d0e4b84a07c4efecadf93b3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.246352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.284273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc3899e9fd44ac08788f6191184bd3b', 'request_id': 'req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.296190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc3899e9fd44ac08788f6191184bd3b', 'request_id': 'req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.307439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc3899e9fd44ac08788f6191184bd3b', 'request_id': 'req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.322997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f8dcac1d0e4b84a07c4efecadf93b3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.325182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] The token cache is being invalidate because project 7fc3899e9fd44ac08788f6191184bd3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:44.325406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.326140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.331168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1102/2201] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:44 2022] DELETE /identity/v3/projects/7fc3899e9fd44ac08788f6191184bd3b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:44.331168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.331168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.335959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f8dcac1d0e4b84a07c4efecadf93b3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.346223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:44.356561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f8dcac1d0e4b84a07c4efecadf93b3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.358051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f8dcac1d0e4b84a07c4efecadf93b3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.374806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c629bf61-7380-41f9-b406-de9dbfb6eead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f8dcac1d0e4b84a07c4efecadf93b3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.387670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1100/2202] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:44 2022] DELETE /identity/v3/users/d9f8dcac1d0e4b84a07c4efecadf93b3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:44.389580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81123b7-980b-4c21-84cb-4aecf9c9fc97 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.390181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81123b7-980b-4c21-84cb-4aecf9c9fc97 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.390703 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81123b7-980b-4c21-84cb-4aecf9c9fc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.442073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655b4b0-5d2d-46f6-9779-428fd3f2fb01 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.451086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b81123b7-980b-4c21-84cb-4aecf9c9fc97 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:44.453440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b81123b7-980b-4c21-84cb-4aecf9c9fc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.549632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:44.588806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.589581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.589946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.590458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.591323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.593943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.620845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1101/2203] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:44.621181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1103/2204] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:44.624059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b19a3-0fee-4f9d-813a-b126cf6578e4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.624866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b19a3-0fee-4f9d-813a-b126cf6578e4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.624988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b19a3-0fee-4f9d-813a-b126cf6578e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.630503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.633448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.652624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3420b6-a4bd-4fb8-b447-0625da84f7db admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.657658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.658341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.658622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.658897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] PATH_INFO: `/v3/projects/dcba4d1cd40444e48c4521475079deb8/users/a9a36f96ab0f4a40857e292a6849737e/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.664233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-699b19a3-0fee-4f9d-813a-b126cf6578e4 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:44.666664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-699b19a3-0fee-4f9d-813a-b126cf6578e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.690057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcba4d1cd40444e48c4521475079deb8, user_id=a9a36f96ab0f4a40857e292a6849737e, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.691887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1102/2205] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:44.696752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.697853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b19a3-0fee-4f9d-813a-b126cf6578e4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.711348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b19a3-0fee-4f9d-813a-b126cf6578e4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.722042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771f15d-4fcd-4467-83a1-648d6a70eadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.722693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f15d-4fcd-4467-83a1-648d6a70eadd admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.722952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f15d-4fcd-4467-83a1-648d6a70eadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.723220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771f15d-4fcd-4467-83a1-648d6a70eadd admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.723944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771f15d-4fcd-4467-83a1-648d6a70eadd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:44.724293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771f15d-4fcd-4467-83a1-648d6a70eadd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.726762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771f15d-4fcd-4467-83a1-648d6a70eadd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.732981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1104/2206] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:44 2022] PUT /identity/v3/projects/dcba4d1cd40444e48c4521475079deb8/users/a9a36f96ab0f4a40857e292a6849737e/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:44.733858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.735546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.762626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1103/2207] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:44.762626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6f0c-ef82-4ac7-9542-3bf6b396f5ef None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.762923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6f0c-ef82-4ac7-9542-3bf6b396f5ef None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.763272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c6f0c-ef82-4ac7-9542-3bf6b396f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.800805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b12c6f0c-ef82-4ac7-9542-3bf6b396f5ef None None] MFA Rules not processed for user `a9a36f96ab0f4a40857e292a6849737e`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:44.802966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b12c6f0c-ef82-4ac7-9542-3bf6b396f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.840965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0a26a-3772-49b0-8f5b-d0bfd6391af6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.848791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:44.849059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:44.849274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:44.849552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] PATH_INFO: `/v3/projects/6e09929bc1514374977d20ecb3078851` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:44.852199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e09929bc1514374977d20ecb3078851)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:44.855973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:44.894052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1104/2208] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:44.897971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6f0c-ef82-4ac7-9542-3bf6b396f5ef None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.909861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c6f0c-ef82-4ac7-9542-3bf6b396f5ef None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:44.922404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e09929bc1514374977d20ecb3078851', 'request_id': 'req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.933383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e09929bc1514374977d20ecb3078851', 'request_id': 'req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.942269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e09929bc1514374977d20ecb3078851', 'request_id': 'req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.969799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] The token cache is being invalidate because project 6e09929bc1514374977d20ecb3078851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:44.970110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.970775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:44.972475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1105/2209] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:44 2022] DELETE /identity/v3/projects/6e09929bc1514374977d20ecb3078851 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:44.973545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:44.974932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:44.984925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:45.004788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.005652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.006057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.006463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.007263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.010563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.035291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:45.049947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1105/2210] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:44 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:45.050832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.052935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:45.076243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3862de3-a4a3-4b38-9ff2-cfd8a9f0382b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.126406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633afbf7-dbb1-4a4f-b0c6-72fc88e6a5dc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.131376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.132030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.132343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.132616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] PATH_INFO: `/v3/projects/990ad881b6be41218782ed5c68bee97e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.133751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990ad881b6be41218782ed5c68bee97e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.136051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.141612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc0d8a-6f4e-4782-aa5c-c3ac11803628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.142322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0d8a-6f4e-4782-aa5c-c3ac11803628 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.142671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0d8a-6f4e-4782-aa5c-c3ac11803628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.143011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc0d8a-6f4e-4782-aa5c-c3ac11803628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.143687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc0d8a-6f4e-4782-aa5c-c3ac11803628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.145678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc0d8a-6f4e-4782-aa5c-c3ac11803628 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.186122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1106/2211] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:44 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:45.187588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.187933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.188309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.193890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990ad881b6be41218782ed5c68bee97e', 'request_id': 'req-c50ca7ff-acdf-429d-96f4-05de322c1ed1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:45.201942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990ad881b6be41218782ed5c68bee97e', 'request_id': 'req-c50ca7ff-acdf-429d-96f4-05de322c1ed1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:45.210784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990ad881b6be41218782ed5c68bee97e', 'request_id': 'req-c50ca7ff-acdf-429d-96f4-05de322c1ed1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:45.211575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:45.213695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:45.227873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] The token cache is being invalidate because project 990ad881b6be41218782ed5c68bee97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:45.228248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:45.229103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:45.232250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1106/2212] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:45 2022] DELETE /identity/v3/projects/990ad881b6be41218782ed5c68bee97e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:45.233246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.234589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:45.360711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1107/2213] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:45.360711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.360711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:45.385368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a263d-65d0-4be8-b7eb-218406bb1e71 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.394967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ca7ff-acdf-429d-96f4-05de322c1ed1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.396293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d69d9-a1f3-4611-8bf2-64819a564767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.397343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d69d9-a1f3-4611-8bf2-64819a564767 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.397662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d69d9-a1f3-4611-8bf2-64819a564767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.398638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d69d9-a1f3-4611-8bf2-64819a564767 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.399444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d69d9-a1f3-4611-8bf2-64819a564767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.402291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6239a6-36a3-4cab-b04b-73701172a4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.402463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d69d9-a1f3-4611-8bf2-64819a564767 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.402805 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6239a6-36a3-4cab-b04b-73701172a4df admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.403087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6239a6-36a3-4cab-b04b-73701172a4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.403363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6239a6-36a3-4cab-b04b-73701172a4df admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.404006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6239a6-36a3-4cab-b04b-73701172a4df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.406013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6239a6-36a3-4cab-b04b-73701172a4df admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.419946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:45.436383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1107/2214] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/users => generated 291 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:45.437780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57141b2-2679-42c3-bd4e-8732399e5033 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.438285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57141b2-2679-42c3-bd4e-8732399e5033 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.438569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57141b2-2679-42c3-bd4e-8732399e5033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.449988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1108/2215] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:45.450735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cdc50-c1f3-4228-8399-ed2ad779de27 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.450896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cdc50-c1f3-4228-8399-ed2ad779de27 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.451395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cdc50-c1f3-4228-8399-ed2ad779de27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.470546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d57141b2-2679-42c3-bd4e-8732399e5033 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:45.471840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d57141b2-2679-42c3-bd4e-8732399e5033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:45.488709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1108/2216] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:45.488709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57141b2-2679-42c3-bd4e-8732399e5033 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.488997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-449cdc50-c1f3-4228-8399-ed2ad779de27 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:45.488997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-449cdc50-c1f3-4228-8399-ed2ad779de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:45.495940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57141b2-2679-42c3-bd4e-8732399e5033 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.502043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4525b-2ef7-48c2-9b6d-dce5c4d70397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.503087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4525b-2ef7-48c2-9b6d-dce5c4d70397 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.503647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4525b-2ef7-48c2-9b6d-dce5c4d70397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.503775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1109/2217] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:45.504262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4525b-2ef7-48c2-9b6d-dce5c4d70397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.505741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4525b-2ef7-48c2-9b6d-dce5c4d70397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.507797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4525b-2ef7-48c2-9b6d-dce5c4d70397 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.510053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cdc50-c1f3-4228-8399-ed2ad779de27 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.526031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cdc50-c1f3-4228-8399-ed2ad779de27 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.530996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1109/2218] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:45.533095 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab723bba-9613-48d0-9912-1eb9db7a0719 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.533566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab723bba-9613-48d0-9912-1eb9db7a0719 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.533997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab723bba-9613-48d0-9912-1eb9db7a0719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.540137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.540969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.541260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.541517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.542193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.546505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.576634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1110/2219] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:45.576634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.590394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ab723bba-9613-48d0-9912-1eb9db7a0719 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:45.591989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ab723bba-9613-48d0-9912-1eb9db7a0719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:45.604007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f61d-7541-430b-a9b9-3b15a18ac395 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.610617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.610822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1110/2220] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:45.611341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.611535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.611777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.611940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab723bba-9613-48d0-9912-1eb9db7a0719 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.612565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:45.612870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.615112 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.626445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab723bba-9613-48d0-9912-1eb9db7a0719 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.635802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1111/2221] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:45.635802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.636203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.636203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.636203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.636203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.636531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.637044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.660384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1111/2222] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:45.660384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.660535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ab346-5fd0-4b02-90b1-b24d6d597527 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.670755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdeaf5-7d92-4494-ac79-6e1b4830e738 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.673584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.674339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.674702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.675035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] PATH_INFO: `/v3/projects/84b5920f88644936b21ec97da0445fbe/users/af0c3ec644404bef83e0557549f72bf8/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.682664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.683556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.683556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.683844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.684535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:45.686260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.686639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.693258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b5920f88644936b21ec97da0445fbe, user_id=af0c3ec644404bef83e0557549f72bf8, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.696038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.702765 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1112/2223] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:45.703832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.715188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:45.715400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074036b-0116-453f-8b2d-46ce97b7948b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.731920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1112/2224] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:45 2022] PUT /identity/v3/projects/84b5920f88644936b21ec97da0445fbe/users/af0c3ec644404bef83e0557549f72bf8/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:45.733342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.831482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.831482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.831482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.831482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.832247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.832247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.877749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1113/2225] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:45.878712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:45.887656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab522e4-aed7-472e-bab7-b4c560c66bc1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.956702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.957618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.957992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.958326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] PATH_INFO: `/v3/projects/375cd031a57847ed99991340bf65d3d3/users/dc83779cc4e54be794873d62e3ac9061/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.960140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfb843-8a55-46ad-94a6-136031d20d9a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:45.967554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:45.968269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:45.968631 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:45.968903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:45.972892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.973617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:45.985831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375cd031a57847ed99991340bf65d3d3, user_id=dc83779cc4e54be794873d62e3ac9061, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:45.988972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.011806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1113/2226] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:45 2022] PUT /identity/v3/projects/375cd031a57847ed99991340bf65d3d3/users/dc83779cc4e54be794873d62e3ac9061/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:46.012747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.030514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1114/2227] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:45 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:46.031551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.062515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:46.120620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b753c8-bfa4-4cbd-91cb-349361a25746 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.133136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b10915-9db5-4e37-afc0-7031d9ec1877 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.139373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44e2de-8935-4915-a9f7-44cc5eb54c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.139373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44e2de-8935-4915-a9f7-44cc5eb54c9e admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.139760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44e2de-8935-4915-a9f7-44cc5eb54c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.139930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44e2de-8935-4915-a9f7-44cc5eb54c9e admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.142083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44e2de-8935-4915-a9f7-44cc5eb54c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.144571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44e2de-8935-4915-a9f7-44cc5eb54c9e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.162156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:46.173296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1115/2228] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:46 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:46.175090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804984-4dc1-4527-ba8a-3c7431792c76 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.175376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804984-4dc1-4527-ba8a-3c7431792c76 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.175645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30804984-4dc1-4527-ba8a-3c7431792c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.189810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.190519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.190862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.191517 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.192562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.194412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.198211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-30804984-4dc1-4527-ba8a-3c7431792c76 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:46.199380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-30804984-4dc1-4527-ba8a-3c7431792c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:46.206526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1114/2229] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:46.207370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1116/2230] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:46.207482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.211238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30804984-4dc1-4527-ba8a-3c7431792c76 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.217666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19e172-c932-4ad9-9645-add79da17697 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.222080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30804984-4dc1-4527-ba8a-3c7431792c76 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.222577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.223963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.224286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.224561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.225632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.228095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.237391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.238812 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.239356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.240003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.240440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.242520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.254106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1117/2231] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:46.255016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.258863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1115/2232] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:46 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:46.260495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.262579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:46.268841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0d9bc-c42f-40e7-b4b5-f74982ef405c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.275996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.277562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.277979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.278669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] PATH_INFO: `/v3/projects/84b5920f88644936b21ec97da0445fbe/users/af0c3ec644404bef83e0557549f72bf8/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.280505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b5920f88644936b21ec97da0445fbe, user_id=af0c3ec644404bef83e0557549f72bf8, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.283434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.296819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1118/2233] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:46 2022] PUT /identity/v3/projects/84b5920f88644936b21ec97da0445fbe/users/af0c3ec644404bef83e0557549f72bf8/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:46.297809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.331504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae27e1-9274-4043-98d1-d16a52ba5231 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.387028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383d47f-f13b-4cd6-b5f5-4799e12ec115 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.392018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.392672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.393009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.393488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.394189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.396251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.404793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a564b-aced-41e7-a508-28666f8636de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.405440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a564b-aced-41e7-a508-28666f8636de service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.405718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a564b-aced-41e7-a508-28666f8636de service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.405991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a564b-aced-41e7-a508-28666f8636de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.411067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1119/2234] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:46.411067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.423292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a564b-aced-41e7-a508-28666f8636de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.423504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765f11f-4fa7-42ac-9bde-3a253923ec94 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.423604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a564b-aced-41e7-a508-28666f8636de service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.428255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.429079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.429429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.429795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.430508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.432353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.435358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1116/2235] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:46.437074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b3c9-86c1-4fc4-b093-c43495c63b78 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.437536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b3c9-86c1-4fc4-b093-c43495c63b78 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.437982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b3c9-86c1-4fc4-b093-c43495c63b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.446365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1120/2236] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:46.446365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.456033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab38ec-6502-4b84-924f-93ea6c0d9585 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.461380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.462028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.462332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.462688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] PATH_INFO: `/v3/projects/0bd6710c4fc140509913dd76d5511dbb/users/91840d1a9c674699b3c18efa664b28dd/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.478680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd6710c4fc140509913dd76d5511dbb, user_id=91840d1a9c674699b3c18efa664b28dd, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.481421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.482477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5c42b3c9-86c1-4fc4-b093-c43495c63b78 None None] MFA Rules not processed for user `af0c3ec644404bef83e0557549f72bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:46.485075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5c42b3c9-86c1-4fc4-b093-c43495c63b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:46.499545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1121/2237] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:46 2022] PUT /identity/v3/projects/0bd6710c4fc140509913dd76d5511dbb/users/91840d1a9c674699b3c18efa664b28dd/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:46.501043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.558029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:46.577868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:46.586311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1117/2238] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:46.586311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d1f80-a4da-4652-beb4-6df02b6b0330 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.586311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d1f80-a4da-4652-beb4-6df02b6b0330 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.586311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d1f80-a4da-4652-beb4-6df02b6b0330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.592836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45e8df-babf-464b-90f7-10307d15a4ca admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.600759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.602139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.602734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.603284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.604544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.605883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.618303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f95d1f80-a4da-4652-beb4-6df02b6b0330 None None] MFA Rules not processed for user `91840d1a9c674699b3c18efa664b28dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:46.619591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f95d1f80-a4da-4652-beb4-6df02b6b0330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:46.623645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1122/2239] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:46.623645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.623645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:46.691599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660aa93-0079-4462-874d-acd7c98acbcd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.695827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc5f0e-fbd6-479e-b8e3-e232861f66f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.696892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc5f0e-fbd6-479e-b8e3-e232861f66f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.697492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc5f0e-fbd6-479e-b8e3-e232861f66f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.697997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc5f0e-fbd6-479e-b8e3-e232861f66f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.702814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1118/2240] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:46.702814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d1f80-a4da-4652-beb4-6df02b6b0330 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.710529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d1f80-a4da-4652-beb4-6df02b6b0330 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.716784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.717412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.717683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.717952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] PATH_INFO: `/v3/projects/85538210a453492e99e3e4548b894a29/users/d94abaf015d14456bef8983726eab392/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.736894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85538210a453492e99e3e4548b894a29, user_id=d94abaf015d14456bef8983726eab392, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.739607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.760506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1119/2241] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:46 2022] PUT /identity/v3/projects/85538210a453492e99e3e4548b894a29/users/d94abaf015d14456bef8983726eab392/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:46.762174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:46.764113 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:46.794985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc5f0e-fbd6-479e-b8e3-e232861f66f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.796854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc5f0e-fbd6-479e-b8e3-e232861f66f4 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.810240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1123/2242] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:46.811777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919e56-f0d4-4552-b44a-2c9917633dcc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.812142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919e56-f0d4-4552-b44a-2c9917633dcc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.812600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919e56-f0d4-4552-b44a-2c9917633dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.844745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797cba6-6620-4edf-a8f6-95499efe052d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:46.847910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-20919e56-f0d4-4552-b44a-2c9917633dcc None None] MFA Rules not processed for user `d94abaf015d14456bef8983726eab392`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:46.849683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:46.850423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-20919e56-f0d4-4552-b44a-2c9917633dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:46.853161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:46.853469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:46.853715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:46.967801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:46.970327 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:46.985576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1120/2243] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:46.998285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1124/2244] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:47.003145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.033836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:47.096277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dacdc-475f-48dd-ab2e-e58892753fb3 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.105665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.106307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.106771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.107058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.107715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.110585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:47.166440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1121/2245] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:47 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:47.173153 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20919e56-f0d4-4552-b44a-2c9917633dcc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.188928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20919e56-f0d4-4552-b44a-2c9917633dcc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.219030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:47.299873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.302854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:47.303584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.304615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.305570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.306086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.306777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.309521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:47.327043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91f86e-880b-4e18-a0f3-a34e1a707b68 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.335472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.336538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.336538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.336782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] PATH_INFO: `/v3/users/99f8c8a010ef43e18fe2d5bd0930fbea` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.343355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f8c8a010ef43e18fe2d5bd0930fbea)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.345384 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:47.345997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1125/2246] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:47 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:47.350142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.363985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768b0f-bf17-4382-b348-27e025c3cf81 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.371786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.372602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.372832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.373244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.374151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.379380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:47.399250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1126/2247] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:47.405017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.418937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af507ee5-7eae-4934-914e-348f362d78a6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.425081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.427111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.427521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.427887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.429234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.431811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:47.439523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f8c8a010ef43e18fe2d5bd0930fbea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:47.440286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f8c8a010ef43e18fe2d5bd0930fbea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:47.445161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1127/2248] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:47.450457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f8c8a010ef43e18fe2d5bd0930fbea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:47.450728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.452949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:47.477446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5ba8d-af0d-4203-b989-dfeb8e6a1b3b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.496721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e06c5a-d6e2-4d12-846c-a537992fa6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.497514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e06c5a-d6e2-4d12-846c-a537992fa6c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.498023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e06c5a-d6e2-4d12-846c-a537992fa6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.498140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e06c5a-d6e2-4d12-846c-a537992fa6c1 admin admin] PATH_INFO: `/v3/projects/ea96f08d4217487a85dcee5d7cae3f36/users/b729eb4287d94786a859c5dd7451245c/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.516189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f8c8a010ef43e18fe2d5bd0930fbea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:47.520520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80841940-04f1-4888-8606-d906ada7c28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f8c8a010ef43e18fe2d5bd0930fbea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:47.526338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e06c5a-d6e2-4d12-846c-a537992fa6c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea96f08d4217487a85dcee5d7cae3f36, user_id=b729eb4287d94786a859c5dd7451245c, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.529677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e06c5a-d6e2-4d12-846c-a537992fa6c1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:47.556460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1122/2249] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:47 2022] DELETE /identity/v3/users/99f8c8a010ef43e18fe2d5bd0930fbea => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:47.559987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.560313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.561492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1128/2250] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:47 2022] PUT /identity/v3/projects/ea96f08d4217487a85dcee5d7cae3f36/users/b729eb4287d94786a859c5dd7451245c/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:47.562136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.572779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e76227-4f7b-4ee7-9196-23021d0607ae None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.573990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e76227-4f7b-4ee7-9196-23021d0607ae None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.575176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e76227-4f7b-4ee7-9196-23021d0607ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.615433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:47.622445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:47.652483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d9e76227-4f7b-4ee7-9196-23021d0607ae None None] MFA Rules not processed for user `b729eb4287d94786a859c5dd7451245c`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:47.653911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d9e76227-4f7b-4ee7-9196-23021d0607ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:47.754194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1129/2251] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:47.761912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e76227-4f7b-4ee7-9196-23021d0607ae None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.807272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1123/2252] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:47.826680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.828139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:47.832720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:47.865351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e76227-4f7b-4ee7-9196-23021d0607ae None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.882260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.883032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.883409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.883758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.884593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.886771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:47.897632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:47.967476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026668-3176-42a5-873d-5a88f3c63fed None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:47.971574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:47.972243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:47.972612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:47.972981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:47.974187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1130/2253] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:47 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:47.975243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:47.984973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:47.985155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:47.986550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.004230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1124/2254] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:48.006353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.028015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b5f62-c53c-4628-a258-6549566542b3 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.098335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdae930-5114-4350-bb5e-91ea09a50a3c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.105807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.106524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.106892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.107450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] PATH_INFO: `/v3/users/a9a36f96ab0f4a40857e292a6849737e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.108799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a36f96ab0f4a40857e292a6849737e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.111009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.130385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11266f02-facd-4964-87d1-5434bd9a91fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.131476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266f02-facd-4964-87d1-5434bd9a91fb admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.131476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266f02-facd-4964-87d1-5434bd9a91fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.131734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266f02-facd-4964-87d1-5434bd9a91fb admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.132365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11266f02-facd-4964-87d1-5434bd9a91fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.134631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11266f02-facd-4964-87d1-5434bd9a91fb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.195310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1125/2255] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:48 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:48.197208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f60d-a33f-4bee-b4ac-e65b03ed4062 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.197459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f60d-a33f-4bee-b4ac-e65b03ed4062 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.197883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843f60d-a33f-4bee-b4ac-e65b03ed4062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.235597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a36f96ab0f4a40857e292a6849737e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:48.243192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c843f60d-a33f-4bee-b4ac-e65b03ed4062 None None] MFA Rules not processed for user `dc83779cc4e54be794873d62e3ac9061`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:48.244398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c843f60d-a33f-4bee-b4ac-e65b03ed4062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:48.254399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a36f96ab0f4a40857e292a6849737e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:48.279557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a36f96ab0f4a40857e292a6849737e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:48.280464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a36f96ab0f4a40857e292a6849737e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:48.297788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a36f96ab0f4a40857e292a6849737e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:48.312244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1131/2256] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:47 2022] DELETE /identity/v3/users/a9a36f96ab0f4a40857e292a6849737e => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:48.313260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.318186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:48.350234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1126/2257] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:48.350234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c2671-60b2-46f3-9efb-6697bfe17076 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.350234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c2671-60b2-46f3-9efb-6697bfe17076 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.350234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c2671-60b2-46f3-9efb-6697bfe17076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.381069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-472c2671-60b2-46f3-9efb-6697bfe17076 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:48.383456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-472c2671-60b2-46f3-9efb-6697bfe17076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:48.452075 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:48.469496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:48.480954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f8917-1248-4940-8f31-e6a6258da834 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.495047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.496125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.496125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.496622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.497375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.499108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.510592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1132/2258] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:48.515716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.517581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:48.530373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1127/2259] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:48.534772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c2671-60b2-46f3-9efb-6697bfe17076 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.560077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c2671-60b2-46f3-9efb-6697bfe17076 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.575132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.575835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.576115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.576409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] PATH_INFO: `/v3/projects/b03781ad40dc4eb0b4daa1f7d5ea81dd/users/0723b38058ad459bb057ab275385e57d/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.586541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb707237-c07d-4631-8bb5-bec852a4168a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.591488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.592487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.592876 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.593252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.608002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03781ad40dc4eb0b4daa1f7d5ea81dd, user_id=0723b38058ad459bb057ab275385e57d, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.612497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.639818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1128/2260] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:48 2022] PUT /identity/v3/projects/b03781ad40dc4eb0b4daa1f7d5ea81dd/users/0723b38058ad459bb057ab275385e57d/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:48.640694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.642605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:48.696833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.698816 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.717021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1133/2261] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:48.717828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.751061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8cb76-3dd1-4152-8498-5a183d54377f service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.803827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3144e93-09ab-4c5a-a970-902e6814e746 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.810740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.811817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.812293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.812715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.890156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.891419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.891860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.892304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.893754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.897490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.924103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1134/2262] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:48.926550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.938680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:48.950470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.954790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:48.973902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b9354-ea96-4bae-9312-d3ddd0578bc9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:48.976530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1129/2263] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:48.978007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:48.990938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:48.991726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:48.991989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:48.992355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] PATH_INFO: `/v3/projects/a8ce63941ef94823a9471a4d9ef5cde4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:48.994297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ce63941ef94823a9471a4d9ef5cde4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:48.999047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4421c70-2421-4271-b991-f7b266b7eac0 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:49.002699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:49.022023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c01989-e50f-4889-ba13-3d7886a2477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:49.022744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c01989-e50f-4889-ba13-3d7886a2477f admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:49.023180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c01989-e50f-4889-ba13-3d7886a2477f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:49.023322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c01989-e50f-4889-ba13-3d7886a2477f admin admin] PATH_INFO: `/v3/projects/b03781ad40dc4eb0b4daa1f7d5ea81dd/users/0723b38058ad459bb057ab275385e57d/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:49.031671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c01989-e50f-4889-ba13-3d7886a2477f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03781ad40dc4eb0b4daa1f7d5ea81dd, user_id=0723b38058ad459bb057ab275385e57d, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:49.034586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c01989-e50f-4889-ba13-3d7886a2477f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:49.079612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1130/2264] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:48 2022] PUT /identity/v3/projects/b03781ad40dc4eb0b4daa1f7d5ea81dd/users/0723b38058ad459bb057ab275385e57d/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:49.092099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:49.092411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:49.092685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:49.167422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:49.191013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] MFA Rules not processed for user `0723b38058ad459bb057ab275385e57d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:49.193947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:49.200826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ce63941ef94823a9471a4d9ef5cde4', 'request_id': 'req-017575db-4b9b-44a2-be74-f399a20affe2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.213379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ce63941ef94823a9471a4d9ef5cde4', 'request_id': 'req-017575db-4b9b-44a2-be74-f399a20affe2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.220694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ce63941ef94823a9471a4d9ef5cde4', 'request_id': 'req-017575db-4b9b-44a2-be74-f399a20affe2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.250348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] The token cache is being invalidate because project a8ce63941ef94823a9471a4d9ef5cde4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:49.250678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.254605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.260268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1135/2265] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:48 2022] DELETE /identity/v3/projects/a8ce63941ef94823a9471a4d9ef5cde4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:49.266959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:49.269847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:49.299428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:49.350718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1131/2266] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:49.368101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:49.369841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:49.460667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017575db-4b9b-44a2-be74-f399a20affe2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:49.464129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34921309-2d34-43fb-ada0-494b96b481b9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:49.470017 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:49.470384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:49.471285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:49.471402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:49.471483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:49.471483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:49.471644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:49.471980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] PATH_INFO: `/v3/users/32caf2fa1d934f3a8aa4ca3a99342adf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:49.473462 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] RBAC: Authorizing `identity:delete_user(user_id=32caf2fa1d934f3a8aa4ca3a99342adf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:49.475562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:49.479246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:49.550038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32caf2fa1d934f3a8aa4ca3a99342adf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.569541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32caf2fa1d934f3a8aa4ca3a99342adf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.585240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:49.587472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:49.593774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32caf2fa1d934f3a8aa4ca3a99342adf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.595219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32caf2fa1d934f3a8aa4ca3a99342adf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.611292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32caf2fa1d934f3a8aa4ca3a99342adf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.613224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1132/2267] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:49.625694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1136/2268] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:49 2022] DELETE /identity/v3/users/32caf2fa1d934f3a8aa4ca3a99342adf => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:49.644740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:49.646332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:49.675787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ea93-156b-41d2-91ed-cced44fa0196 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:49.810547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:49.810547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:49.810547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:49.811903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] PATH_INFO: `/v3/projects/dcba4d1cd40444e48c4521475079deb8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:49.813728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcba4d1cd40444e48c4521475079deb8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:49.818664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:49.902982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:49.933018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcba4d1cd40444e48c4521475079deb8', 'request_id': 'req-5e0e4842-3b63-4be8-b167-08863df556e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.940036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcba4d1cd40444e48c4521475079deb8', 'request_id': 'req-5e0e4842-3b63-4be8-b167-08863df556e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.957238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcba4d1cd40444e48c4521475079deb8', 'request_id': 'req-5e0e4842-3b63-4be8-b167-08863df556e5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.985833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] The token cache is being invalidate because project dcba4d1cd40444e48c4521475079deb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:49.987695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.991395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:49.994287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1133/2269] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:49 2022] DELETE /identity/v3/projects/dcba4d1cd40444e48c4521475079deb8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:49.998513 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:50.000590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:50.026624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e9137-cecb-43f0-9d10-dc71c211161e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:50.106246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:50.137183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:50.137854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:50.138136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:50.138411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] PATH_INFO: `/v3/users/af0c3ec644404bef83e0557549f72bf8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:50.140230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0c3ec644404bef83e0557549f72bf8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:50.142332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:50.264765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:50.266320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:50.332687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0c3ec644404bef83e0557549f72bf8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.333055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e4842-3b63-4be8-b167-08863df556e5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:50.343072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:50.345014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:50.345382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:50.345736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0c3ec644404bef83e0557549f72bf8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.346313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] PATH_INFO: `/v3/projects/ee5c4704d30f48a5b7800e6a31715b71` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:50.347724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee5c4704d30f48a5b7800e6a31715b71)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:50.366296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0c3ec644404bef83e0557549f72bf8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.367182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0c3ec644404bef83e0557549f72bf8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.369311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:50.378514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-67ade055-d08f-4a6f-9514-7993b091780d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0c3ec644404bef83e0557549f72bf8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.388289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1137/2270] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:49 2022] DELETE /identity/v3/users/af0c3ec644404bef83e0557549f72bf8 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:50.468978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:50.469274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:50.469561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:50.489294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee5c4704d30f48a5b7800e6a31715b71', 'request_id': 'req-4584d076-0a03-4559-9bfa-c7046fe806dc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.496011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] MFA Rules not processed for user `dc83779cc4e54be794873d62e3ac9061`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:50.497370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:50.500982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee5c4704d30f48a5b7800e6a31715b71', 'request_id': 'req-4584d076-0a03-4559-9bfa-c7046fe806dc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.511963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee5c4704d30f48a5b7800e6a31715b71', 'request_id': 'req-4584d076-0a03-4559-9bfa-c7046fe806dc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.534644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] The token cache is being invalidate because project ee5c4704d30f48a5b7800e6a31715b71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:50.534964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.536316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.541723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1134/2271] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:50 2022] DELETE /identity/v3/projects/ee5c4704d30f48a5b7800e6a31715b71 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:50.546569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:50.549628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:50.598329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1138/2272] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:50.677691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584d076-0a03-4559-9bfa-c7046fe806dc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:50.687649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:50.688358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:50.688527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:50.688803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] PATH_INFO: `/v3/users/59fdd66ccd314edcb87d802e2273702b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:50.690103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=59fdd66ccd314edcb87d802e2273702b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:50.692900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:50.791025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59fdd66ccd314edcb87d802e2273702b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.792140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59fdd66ccd314edcb87d802e2273702b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.800138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:50.813536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59fdd66ccd314edcb87d802e2273702b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.840478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59fdd66ccd314edcb87d802e2273702b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.842089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59fdd66ccd314edcb87d802e2273702b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:50.875457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1135/2273] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:50 2022] DELETE /identity/v3/users/59fdd66ccd314edcb87d802e2273702b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:51.115130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.116710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.145919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac82e4-dea2-4fca-855a-2d418fecc583 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.220695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.221353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.221633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.221905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] PATH_INFO: `/v3/projects/84b5920f88644936b21ec97da0445fbe` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.223216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b5920f88644936b21ec97da0445fbe)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.225497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.233413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:51.307798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b5920f88644936b21ec97da0445fbe', 'request_id': 'req-ef52eb92-8ec5-4ef1-89f0-449293fece7d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.316018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b5920f88644936b21ec97da0445fbe', 'request_id': 'req-ef52eb92-8ec5-4ef1-89f0-449293fece7d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.333587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b5920f88644936b21ec97da0445fbe', 'request_id': 'req-ef52eb92-8ec5-4ef1-89f0-449293fece7d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.352856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.354962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.360075 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] The token cache is being invalidate because project 84b5920f88644936b21ec97da0445fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:51.360658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.361678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ef52eb92-8ec5-4ef1-89f0-449293fece7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.365354 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1139/2274] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:51 2022] DELETE /identity/v3/projects/84b5920f88644936b21ec97da0445fbe => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:51.379720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c19fbe-e47c-4370-a62c-d13a9c47ba6d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.396187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315e873-1c0d-41d7-b9c2-02f11488d58f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.396478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315e873-1c0d-41d7-b9c2-02f11488d58f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.396761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315e873-1c0d-41d7-b9c2-02f11488d58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.436874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f315e873-1c0d-41d7-b9c2-02f11488d58f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:51.438400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f315e873-1c0d-41d7-b9c2-02f11488d58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.459777 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.460516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.461182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.461572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] PATH_INFO: `/v3/projects/2e03421e2c1f43b599e6d7a03c0c6576` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.462982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e03421e2c1f43b599e6d7a03c0c6576)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.465435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.519807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1140/2275] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:51.524620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e03421e2c1f43b599e6d7a03c0c6576', 'request_id': 'req-fc6549fe-08bf-4055-a464-fe843b70fdfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.529090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428143a-bdbb-4628-85d1-f14c925aa6dc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.529514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428143a-bdbb-4628-85d1-f14c925aa6dc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.529893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428143a-bdbb-4628-85d1-f14c925aa6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.533483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e03421e2c1f43b599e6d7a03c0c6576', 'request_id': 'req-fc6549fe-08bf-4055-a464-fe843b70fdfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.542052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e03421e2c1f43b599e6d7a03c0c6576', 'request_id': 'req-fc6549fe-08bf-4055-a464-fe843b70fdfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.554306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4428143a-bdbb-4628-85d1-f14c925aa6dc None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:51.556093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4428143a-bdbb-4628-85d1-f14c925aa6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.568769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] The token cache is being invalidate because project 2e03421e2c1f43b599e6d7a03c0c6576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:51.569079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.576816 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc6549fe-08bf-4055-a464-fe843b70fdfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:51.589404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1136/2276] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:51 2022] DELETE /identity/v3/projects/2e03421e2c1f43b599e6d7a03c0c6576 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:51.595306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.595591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.595855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.629378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:51.631078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.632099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1141/2277] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:51.633188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428143a-bdbb-4628-85d1-f14c925aa6dc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.640801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1137/2278] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:51.641852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.643134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.644006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428143a-bdbb-4628-85d1-f14c925aa6dc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.649046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e254270-049e-4842-85aa-6dba78d773d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.650039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e254270-049e-4842-85aa-6dba78d773d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.650356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e254270-049e-4842-85aa-6dba78d773d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.651390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e254270-049e-4842-85aa-6dba78d773d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.652055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e254270-049e-4842-85aa-6dba78d773d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:51.652412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e254270-049e-4842-85aa-6dba78d773d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.654338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e254270-049e-4842-85aa-6dba78d773d3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.666261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1142/2279] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:51.667534 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b35fd8-3990-4434-833d-173a70960ea6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.667854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b35fd8-3990-4434-833d-173a70960ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.668101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b35fd8-3990-4434-833d-173a70960ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.689451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-05b35fd8-3990-4434-833d-173a70960ea6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:51.690736 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-05b35fd8-3990-4434-833d-173a70960ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.699403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1143/2280] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:51.700331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b35fd8-3990-4434-833d-173a70960ea6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.708182 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:51.711378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b35fd8-3990-4434-833d-173a70960ea6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.713260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70993999-14c0-4968-9a79-ce875e8c96f7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.717089 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.717789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.718260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.718541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.718814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.718929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.719419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.719528 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.720786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.721412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.723625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:51.737451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:51.767052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1144/2281] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:51.767052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.813459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.814617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.825034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1138/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:51.825935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.838092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb5ca-af59-4d3a-a93a-258eb72cfcbb service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.854644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28eef1-60e5-4b3d-9e17-6647fce1fffa admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.861353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.862299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.862587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.862864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.863590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:51.863903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.865752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.891904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1145/2283] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:03:51.893516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:51.909210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145045a7-7186-41a5-b2f6-4dd827c0f492 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:51.919126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.919869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.920170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.920496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.921970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.924832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.939856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:51.941360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:51.942533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:51.943411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:51.944491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:51.947032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:51.963785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1146/2284] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:51.969025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.010541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1139/2285] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:52.015330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.034339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875c2c5-e73e-426a-9202-98a99436fcf2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.104688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed1661-d1e2-42d6-b7fc-eb789718589e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.118084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.120921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.121203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.121479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.122129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.124087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.158403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.159339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.159739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.160116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.160976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.172651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1147/2286] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:51 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:52.173423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.174115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.175377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:52.193479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1140/2287] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:52.193479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.194683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70867d0-db39-4c4a-9471-29ec21b3aaec admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.201474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.202413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.203345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.203679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] PATH_INFO: `/v3/users/117a819351b742deb901c081d0a4347b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.208729 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8278d-d2bd-403c-8c61-9a9e9a87d644 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.215714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.218332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.218625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.224763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] RBAC: Authorizing `identity:delete_user(user_id=117a819351b742deb901c081d0a4347b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.225630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.227758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.228365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.230117 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.258555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1141/2288] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:52.258555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.280014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e8f68-af09-4239-a737-c672c0c2b2d3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.295686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.296630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.297149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.297451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.298180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.299859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.329155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1142/2289] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:52.329155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.340420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e34415-5e10-4b7b-9ae0-afde941a876c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.346639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.347486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.347985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.348360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.348645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '117a819351b742deb901c081d0a4347b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:52.349155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.351648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.352872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=6,keystone:SELECT=87,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:52.354949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:52.365584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '117a819351b742deb901c081d0a4347b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:52.375935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1143/2290] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:52.376930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.390040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc85975-4f83-471f-bff6-95215890bf43 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.396541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779468f-4455-46e7-a312-243996c26878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.397398 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779468f-4455-46e7-a312-243996c26878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.397650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779468f-4455-46e7-a312-243996c26878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.398015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779468f-4455-46e7-a312-243996c26878 admin admin] PATH_INFO: `/v3/projects/b7d624d60c47492e8aeda03a32c6d84b/users/b0bd7cbe6cbe4019ba478b57c9ae94d4/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.407587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '117a819351b742deb901c081d0a4347b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:52.408522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '117a819351b742deb901c081d0a4347b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:52.415696 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779468f-4455-46e7-a312-243996c26878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d624d60c47492e8aeda03a32c6d84b, user_id=b0bd7cbe6cbe4019ba478b57c9ae94d4, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.419740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779468f-4455-46e7-a312-243996c26878 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.420822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '117a819351b742deb901c081d0a4347b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:52.429770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1148/2291] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:03:52 2022] DELETE /identity/v3/users/117a819351b742deb901c081d0a4347b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:52.434550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.435921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:52.440717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1144/2292] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:52 2022] PUT /identity/v3/projects/b7d624d60c47492e8aeda03a32c6d84b/users/b0bd7cbe6cbe4019ba478b57c9ae94d4/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:52.448830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.449283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.451223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.478188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:52.481190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:52.582649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1145/2293] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:52.587404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd82af-3cba-4073-bf47-8905d2d9814e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.593293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.595424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:52.597689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0d9a7-6c97-424e-9b59-17b3b17ec7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.599503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0d9a7-6c97-424e-9b59-17b3b17ec7dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.600282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0d9a7-6c97-424e-9b59-17b3b17ec7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.600526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0d9a7-6c97-424e-9b59-17b3b17ec7dd admin admin] PATH_INFO: `/v3/projects/b38c1e0ae4604bc090ea3793361fcbf6/users/9b1c7aa5819e4e5581c9c5595a75017e/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.622605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0d9a7-6c97-424e-9b59-17b3b17ec7dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38c1e0ae4604bc090ea3793361fcbf6, user_id=9b1c7aa5819e4e5581c9c5595a75017e, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.627041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0d9a7-6c97-424e-9b59-17b3b17ec7dd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.655677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1149/2294] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:52 2022] PUT /identity/v3/projects/b38c1e0ae4604bc090ea3793361fcbf6/users/9b1c7aa5819e4e5581c9c5595a75017e/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:52.671882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.672197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.672481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.701436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87c72-7da9-4cca-80fc-15f161bb1cd3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.728173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] MFA Rules not processed for user `9b1c7aa5819e4e5581c9c5595a75017e`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:52.728173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:52.785575 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.786443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.786724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.786995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.837294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:52.858145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1150/2295] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:52.859188 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.860544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:52.861567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.863856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.877391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1146/2296] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:52.878260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:52.879750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:52.903255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54116f3c-9e83-4ab4-b425-f92646e178c2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:52.915186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:52.916102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:52.916470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:52.916717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] PATH_INFO: `/v3/projects/3fba05bd3cd4401597809b1fd0cdb439` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:52.918294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fba05bd3cd4401597809b1fd0cdb439)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:52.920804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:52.975410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:52.993972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fba05bd3cd4401597809b1fd0cdb439', 'request_id': 'req-d2627ab2-7305-4fa3-ad3e-324c27bef41a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:52.996793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d212ed-ee63-4b76-91cb-e786083faab7 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:53.006641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fba05bd3cd4401597809b1fd0cdb439', 'request_id': 'req-d2627ab2-7305-4fa3-ad3e-324c27bef41a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:53.015173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fba05bd3cd4401597809b1fd0cdb439', 'request_id': 'req-d2627ab2-7305-4fa3-ad3e-324c27bef41a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:53.035010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] The token cache is being invalidate because project 3fba05bd3cd4401597809b1fd0cdb439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:03:53.035377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:53.037108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:03:53.043557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1151/2297] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:03:52 2022] DELETE /identity/v3/projects/3fba05bd3cd4401597809b1fd0cdb439 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:53.044696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:53.046216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:53.064896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a085079-4352-4225-8f1a-a8d928609afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:53.065533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a085079-4352-4225-8f1a-a8d928609afd service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.065812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a085079-4352-4225-8f1a-a8d928609afd service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.066084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a085079-4352-4225-8f1a-a8d928609afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.068128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4a085079-4352-4225-8f1a-a8d928609afd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:53.156834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a085079-4352-4225-8f1a-a8d928609afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:53.158572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a085079-4352-4225-8f1a-a8d928609afd service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:53.165692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2627ab2-7305-4fa3-ad3e-324c27bef41a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:53.171024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1147/2298] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:53.172568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f0b4a-7178-4e35-8399-3cb69da6f033 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.173031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f0b4a-7178-4e35-8399-3cb69da6f033 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.173208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f0b4a-7178-4e35-8399-3cb69da6f033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.183374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db7f85-1a8c-43a4-b327-b3391ce68af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:53.184813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db7f85-1a8c-43a4-b327-b3391ce68af7 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.185102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db7f85-1a8c-43a4-b327-b3391ce68af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.185378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db7f85-1a8c-43a4-b327-b3391ce68af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.187227 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-18db7f85-1a8c-43a4-b327-b3391ce68af7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:53.198369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-970f0b4a-7178-4e35-8399-3cb69da6f033 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:53.200454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-970f0b4a-7178-4e35-8399-3cb69da6f033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:53.269484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db7f85-1a8c-43a4-b327-b3391ce68af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:53.271238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db7f85-1a8c-43a4-b327-b3391ce68af7 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:53.283705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1152/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:53.289098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:53.341697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1148/2300] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:53.349342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348eb48-008c-483c-9f4f-a24dc3c7bc20 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.349629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348eb48-008c-483c-9f4f-a24dc3c7bc20 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.349894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348eb48-008c-483c-9f4f-a24dc3c7bc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.399752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6348eb48-008c-483c-9f4f-a24dc3c7bc20 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:53.401393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6348eb48-008c-483c-9f4f-a24dc3c7bc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:53.438737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1153/2301] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:53.442253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f0b4a-7178-4e35-8399-3cb69da6f033 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:53.455085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f0b4a-7178-4e35-8399-3cb69da6f033 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:53.468602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:53.469242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.469521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.470708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.472971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:03:53.473305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:53.480675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:53.517771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1149/2302] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:03:53.523516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348eb48-008c-483c-9f4f-a24dc3c7bc20 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:53.537502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348eb48-008c-483c-9f4f-a24dc3c7bc20 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:53.550716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:53.551364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.551663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.551923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.556295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:53.561718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:53.614700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1154/2303] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:53 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:53.617155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:53.659703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec67203-4774-49fc-a073-39872375eb51 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:53.842724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0139e9c-d06b-4ae8-99b3-7f57807339f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:53.844170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139e9c-d06b-4ae8-99b3-7f57807339f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.844459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139e9c-d06b-4ae8-99b3-7f57807339f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.844827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139e9c-d06b-4ae8-99b3-7f57807339f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.845548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0139e9c-d06b-4ae8-99b3-7f57807339f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:53.848057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0139e9c-d06b-4ae8-99b3-7f57807339f2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:53.907836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1150/2304] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:53 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:53.909616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:53.924456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3b9c3-098d-4d67-b0be-0084704f3635 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:53.936924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:53.937791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.938231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.939357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.940526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:53.941660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:53.941955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:53.942231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:53.944019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:53.975171 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1155/2305] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:53.980345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:53.999552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0239d8a-bf57-44ec-bef8-467ed873e69b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.011203 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] MFA Rules not processed for user `b0bd7cbe6cbe4019ba478b57c9ae94d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:54.014686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:54.016958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.017931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.017931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.018645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.019533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.021343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.047821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1156/2306] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:54.049652 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.079167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55745b86-6f08-4eb5-b73c-8a46c6da1529 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.086861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343eaf7-3552-4881-974b-fefc66d7b147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.087786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343eaf7-3552-4881-974b-fefc66d7b147 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.089570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343eaf7-3552-4881-974b-fefc66d7b147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.089570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343eaf7-3552-4881-974b-fefc66d7b147 admin admin] PATH_INFO: `/v3/projects/c6ed4f1f1246408099252ebea491832c/users/2b81fb00b1b545308d01f74a55b7e77b/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.111522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343eaf7-3552-4881-974b-fefc66d7b147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ed4f1f1246408099252ebea491832c, user_id=2b81fb00b1b545308d01f74a55b7e77b, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.116451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343eaf7-3552-4881-974b-fefc66d7b147 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.172351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1157/2307] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:54 2022] PUT /identity/v3/projects/c6ed4f1f1246408099252ebea491832c/users/2b81fb00b1b545308d01f74a55b7e77b/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:03:54.177554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1151/2308] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:54.178958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.179597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.180127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.183047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.187138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:54.215571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:54.220863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce607c7-224c-4d7b-b55c-175274636646 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.249861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:54.252251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:54.306558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:54.323579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.324298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.324513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.324915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.325558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.327611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.379493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1152/2309] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:03:54 2022] POST /identity/v3/projects => generated 371 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:03:54.382935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.398072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1158/2310] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:03:54.410957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.412470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:54.478004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f296d4-1de3-4c82-b44b-3db1cef4c4ca admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.483464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.484289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.485057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.485475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.486621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.487235 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c6b9c-729c-40bc-b295-b9ff4bdc8912 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.489296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.519621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1153/2311] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:03:54 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:03:54.524056 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.536986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14bf870-4907-4867-b645-31017e903054 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.543478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.544171 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.544548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.544917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.545770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.547678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.558243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.559014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.559421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.559788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.569205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.571076 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.574559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1154/2312] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:54.579267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.582761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1159/2313] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:03:54.594596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca4a52-cf3e-49fd-89c8-ae2474dae4ff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.601496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.603106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.603480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.603814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] PATH_INFO: `/v3/projects/fd657dea7ffa46138601eec429d624f3/users/36c7111e5b0c45609cc615258c8a30ea/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.622616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd657dea7ffa46138601eec429d624f3, user_id=36c7111e5b0c45609cc615258c8a30ea, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.626178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.643047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1155/2314] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:54 2022] PUT /identity/v3/projects/fd657dea7ffa46138601eec429d624f3/users/36c7111e5b0c45609cc615258c8a30ea/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:54.647517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.658379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ef8cb-e1f4-4f8c-9531-5b58309db93b service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.739272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285fb0fd-c3d8-4950-b7d1-2732cfabee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.740022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fb0fd-c3d8-4950-b7d1-2732cfabee52 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.740408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fb0fd-c3d8-4950-b7d1-2732cfabee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.740751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285fb0fd-c3d8-4950-b7d1-2732cfabee52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.741460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fb0fd-c3d8-4950-b7d1-2732cfabee52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.743758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285fb0fd-c3d8-4950-b7d1-2732cfabee52 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.747750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:54.773804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1160/2315] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:03:54.778344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:54.790285 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4b8bb-7607-42d7-aa55-c6be8b74764f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:54.799180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:54.800101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:54.800504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:54.800926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] PATH_INFO: `/v3/projects/fd657dea7ffa46138601eec429d624f3/users/36c7111e5b0c45609cc615258c8a30ea/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:54.802984 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd657dea7ffa46138601eec429d624f3, user_id=36c7111e5b0c45609cc615258c8a30ea, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:54.806172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:54.819037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1156/2316] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:03:54 2022] PUT /identity/v3/projects/fd657dea7ffa46138601eec429d624f3/users/36c7111e5b0c45609cc615258c8a30ea/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:03:55.822382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19362d4f-3b6d-431a-8224-41c538c7c03a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:55.822925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19362d4f-3b6d-431a-8224-41c538c7c03a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:55.823192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19362d4f-3b6d-431a-8224-41c538c7c03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:55.859909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-19362d4f-3b6d-431a-8224-41c538c7c03a None None] MFA Rules not processed for user `36c7111e5b0c45609cc615258c8a30ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:55.861994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-19362d4f-3b6d-431a-8224-41c538c7c03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:56.029461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1161/2317] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:56.042214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:56.045337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:56.130208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f05d78-7e40-4717-b6b2-aa19b6c5772c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:56.134556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:56.135572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:56.136004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:56.136454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:56.173609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:03:56.207893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:56.209307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:56.249986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1157/2318] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:03:59.519271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c0b8f-a135-4a8c-8d15-fcc4bbe04b87 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:59.520754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c0b8f-a135-4a8c-8d15-fcc4bbe04b87 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:59.521229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147c0b8f-a135-4a8c-8d15-fcc4bbe04b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:59.554108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-147c0b8f-a135-4a8c-8d15-fcc4bbe04b87 None None] MFA Rules not processed for user `2b81fb00b1b545308d01f74a55b7e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:03:59.555667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-147c0b8f-a135-4a8c-8d15-fcc4bbe04b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:59.623139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1162/2319] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:03:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:03:59.638179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:03:59.639743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:03:59.738527 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3890c9-873a-4ee3-9324-99a96aff8f4e service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:03:59.747845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:03:59.748705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:03:59.749600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:03:59.749951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:03:59.760634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:03:59.762230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:03:59.771588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1158/2320] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:03:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:01.062831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c0b8f-a135-4a8c-8d15-fcc4bbe04b87 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:01.088273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147c0b8f-a135-4a8c-8d15-fcc4bbe04b87 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:01.097412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444737b-6398-4fc5-b593-c97c7d4005c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:01.098393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444737b-6398-4fc5-b593-c97c7d4005c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:01.098944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444737b-6398-4fc5-b593-c97c7d4005c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:01.099560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444737b-6398-4fc5-b593-c97c7d4005c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:01.120925 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444737b-6398-4fc5-b593-c97c7d4005c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:01.122407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444737b-6398-4fc5-b593-c97c7d4005c1 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:01.140958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1163/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:01.963495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:01.964978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:02.042748 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474aac91-1cde-4c47-bc1d-1e678de8763d service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:02.048388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:02.049179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:02.049668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:02.050901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] PATH_INFO: `/v3/users/d94abaf015d14456bef8983726eab392` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:02.052346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d94abaf015d14456bef8983726eab392)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:02.054697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:02.204855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd94abaf015d14456bef8983726eab392'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:02.207267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd94abaf015d14456bef8983726eab392'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:02.228483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd94abaf015d14456bef8983726eab392'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:02.238845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:02.260509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd94abaf015d14456bef8983726eab392'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:02.262449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd94abaf015d14456bef8983726eab392'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:02.326829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1159/2322] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:04:01 2022] DELETE /identity/v3/users/d94abaf015d14456bef8983726eab392 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:02.329459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7e51-32d7-42f3-9edc-14a3546a2ae1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:02.329952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7e51-32d7-42f3-9edc-14a3546a2ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:02.330037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d7e51-32d7-42f3-9edc-14a3546a2ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:02.368541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-606d7e51-32d7-42f3-9edc-14a3546a2ae1 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:02.385904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-606d7e51-32d7-42f3-9edc-14a3546a2ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:02.563424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:02.585884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1164/2323] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:02.596911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:02.598550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:02.688127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ebf51-9997-40f3-b5d3-4794faed30fc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:02.695566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:02.698225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:02.701427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:02.701796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:02.723866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:02.725400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:02.749728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1160/2324] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:02.855531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07801377-ae42-4bc1-a79b-7d8e098c2f03 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:02.855888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07801377-ae42-4bc1-a79b-7d8e098c2f03 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:02.856280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07801377-ae42-4bc1-a79b-7d8e098c2f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:02.924507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-07801377-ae42-4bc1-a79b-7d8e098c2f03 None None] MFA Rules not processed for user `2b81fb00b1b545308d01f74a55b7e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:02.925954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-07801377-ae42-4bc1-a79b-7d8e098c2f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:03.080955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1165/2325] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:03.508445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:03.511869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:03.537690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4735f-ea9b-4f8a-b833-e1b096379cd4 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:03.547997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:03.549671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:03.549956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:03.550231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] PATH_INFO: `/v3/projects/85538210a453492e99e3e4548b894a29` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:03.557053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85538210a453492e99e3e4548b894a29)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:03.558950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:03.655291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85538210a453492e99e3e4548b894a29', 'request_id': 'req-4c96d7de-a6e1-44a5-b478-7406135cba6c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.661597 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85538210a453492e99e3e4548b894a29', 'request_id': 'req-4c96d7de-a6e1-44a5-b478-7406135cba6c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.671459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85538210a453492e99e3e4548b894a29', 'request_id': 'req-4c96d7de-a6e1-44a5-b478-7406135cba6c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.688778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] The token cache is being invalidate because project 85538210a453492e99e3e4548b894a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:04:03.689149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.689868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.691478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1161/2326] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:04:03 2022] DELETE /identity/v3/projects/85538210a453492e99e3e4548b894a29 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:03.695694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07801377-ae42-4bc1-a79b-7d8e098c2f03 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:03.708956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07801377-ae42-4bc1-a79b-7d8e098c2f03 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:03.802248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:03.802962 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:03.804662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:03.805032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] PATH_INFO: `/v3/users/b729eb4287d94786a859c5dd7451245c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:03.807864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b729eb4287d94786a859c5dd7451245c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:03.811413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:03.918859 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b729eb4287d94786a859c5dd7451245c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.931330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b729eb4287d94786a859c5dd7451245c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.947642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b729eb4287d94786a859c5dd7451245c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.948344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b729eb4287d94786a859c5dd7451245c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.957841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b729eb4287d94786a859c5dd7451245c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:03.965558 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1166/2327] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:04:03 2022] DELETE /identity/v3/users/b729eb4287d94786a859c5dd7451245c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:04.844713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:04.846308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:04.869167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96d7de-a6e1-44a5-b478-7406135cba6c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:05.003945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:05.004982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:05.005265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:05.005536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] PATH_INFO: `/v3/projects/ea96f08d4217487a85dcee5d7cae3f36` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:05.006821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea96f08d4217487a85dcee5d7cae3f36)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:05.011199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:05.017096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:05.099438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea96f08d4217487a85dcee5d7cae3f36', 'request_id': 'req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.108719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea96f08d4217487a85dcee5d7cae3f36', 'request_id': 'req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.117522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea96f08d4217487a85dcee5d7cae3f36', 'request_id': 'req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.159720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] The token cache is being invalidate because project ea96f08d4217487a85dcee5d7cae3f36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:04:05.160027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.160831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.163177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1162/2328] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:04:04 2022] DELETE /identity/v3/projects/ea96f08d4217487a85dcee5d7cae3f36 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:05.169471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:05.176173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:05.218783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b09483-ec5d-40ba-b936-0fde231fecc2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:05.266185 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:05.327951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:05.329063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:05.329901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:05.330482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] PATH_INFO: `/v3/users/0723b38058ad459bb057ab275385e57d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:05.332283 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0723b38058ad459bb057ab275385e57d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:05.335653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:05.458281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0723b38058ad459bb057ab275385e57d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.473306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0723b38058ad459bb057ab275385e57d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.499291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0723b38058ad459bb057ab275385e57d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.500044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0723b38058ad459bb057ab275385e57d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.514333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6023a2eb-bb32-41fb-a534-e3add99af2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0723b38058ad459bb057ab275385e57d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:05.530036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1167/2329] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:04:05 2022] DELETE /identity/v3/users/0723b38058ad459bb057ab275385e57d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:05.909899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:05.911798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:05.934643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af5d4d-dfe6-4624-8b1e-a037edf0bb51 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:06.014189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:06.015032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:06.015386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:06.015707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] PATH_INFO: `/v3/projects/b03781ad40dc4eb0b4daa1f7d5ea81dd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:06.016979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03781ad40dc4eb0b4daa1f7d5ea81dd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:06.019125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:06.078619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03781ad40dc4eb0b4daa1f7d5ea81dd', 'request_id': 'req-3b621fdf-8925-46a6-acd1-c33b6eb57a99'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:06.085496 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03781ad40dc4eb0b4daa1f7d5ea81dd', 'request_id': 'req-3b621fdf-8925-46a6-acd1-c33b6eb57a99'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:06.110664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03781ad40dc4eb0b4daa1f7d5ea81dd', 'request_id': 'req-3b621fdf-8925-46a6-acd1-c33b6eb57a99'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:06.137588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] The token cache is being invalidate because project b03781ad40dc4eb0b4daa1f7d5ea81dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:04:06.137588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:06.139322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:06.141353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1163/2330] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:04:05 2022] DELETE /identity/v3/projects/b03781ad40dc4eb0b4daa1f7d5ea81dd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:09.047077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451e924-f55d-4d32-9494-7c1c7ec0a45c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:09.047662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451e924-f55d-4d32-9494-7c1c7ec0a45c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:09.047662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451e924-f55d-4d32-9494-7c1c7ec0a45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:09.073032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8451e924-f55d-4d32-9494-7c1c7ec0a45c None None] MFA Rules not processed for user `06e60909a8b04008af659620527e3236`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:09.074535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8451e924-f55d-4d32-9494-7c1c7ec0a45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:09.148568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1168/2331] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:12.407080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:12.408730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:12.436630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:12.483918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b621fdf-8925-46a6-acd1-c33b6eb57a99 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:12.490044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:12.490688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:12.490964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:12.491234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] PATH_INFO: `/v3/users/9b1c7aa5819e4e5581c9c5595a75017e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:12.492457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b1c7aa5819e4e5581c9c5595a75017e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:12.494544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:12.547506 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b1c7aa5819e4e5581c9c5595a75017e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:12.548198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b1c7aa5819e4e5581c9c5595a75017e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:12.557362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b1c7aa5819e4e5581c9c5595a75017e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:12.569501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b1c7aa5819e4e5581c9c5595a75017e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:12.570166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b1c7aa5819e4e5581c9c5595a75017e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:12.583588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1164/2332] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:04:12 2022] DELETE /identity/v3/users/9b1c7aa5819e4e5581c9c5595a75017e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:12.589048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:12.589629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:12.589629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:12.611143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:12.612407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:12.669465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:12.687734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1169/2333] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:12.699385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:12.700887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:12.770283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e87976-61d1-4393-bdc8-4ab294514884 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:12.774019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4f16e-8a23-43a1-aeee-189fff57ade9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:12.774857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4f16e-8a23-43a1-aeee-189fff57ade9 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:12.775245 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4f16e-8a23-43a1-aeee-189fff57ade9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:12.775522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4f16e-8a23-43a1-aeee-189fff57ade9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:12.785197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4f16e-8a23-43a1-aeee-189fff57ade9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:12.786634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4f16e-8a23-43a1-aeee-189fff57ade9 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:12.797558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1165/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:13.169865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.171733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:13.187650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e07085-8a2d-4ac1-86a1-890413de3297 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.193595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.194376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.195212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.195492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] PATH_INFO: `/v3/projects/b38c1e0ae4604bc090ea3793361fcbf6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.206719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38c1e0ae4604bc090ea3793361fcbf6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.210706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.270311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38c1e0ae4604bc090ea3793361fcbf6', 'request_id': 'req-cdfd3525-8d63-4eac-874c-7c52319a7486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:13.277145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38c1e0ae4604bc090ea3793361fcbf6', 'request_id': 'req-cdfd3525-8d63-4eac-874c-7c52319a7486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:13.283371 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38c1e0ae4604bc090ea3793361fcbf6', 'request_id': 'req-cdfd3525-8d63-4eac-874c-7c52319a7486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:13.299025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] The token cache is being invalidate because project b38c1e0ae4604bc090ea3793361fcbf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:04:13.299318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:13.299939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cdfd3525-8d63-4eac-874c-7c52319a7486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:13.301678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1170/2335] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:04:13 2022] DELETE /identity/v3/projects/b38c1e0ae4604bc090ea3793361fcbf6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:13.306861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113a5aa-2ab6-4dfc-83f4-e002066d0df9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.307461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113a5aa-2ab6-4dfc-83f4-e002066d0df9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.307461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113a5aa-2ab6-4dfc-83f4-e002066d0df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.327202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c113a5aa-2ab6-4dfc-83f4-e002066d0df9 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:13.328616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c113a5aa-2ab6-4dfc-83f4-e002066d0df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:13.373612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:13.400979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1166/2336] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:13.409080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd818d-8b1f-45fe-bbef-16fd9b23af2f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.409368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd818d-8b1f-45fe-bbef-16fd9b23af2f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.409639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd818d-8b1f-45fe-bbef-16fd9b23af2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.431251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c3bd818d-8b1f-45fe-bbef-16fd9b23af2f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:13.432505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c3bd818d-8b1f-45fe-bbef-16fd9b23af2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:13.441401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1171/2337] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:13.446439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113a5aa-2ab6-4dfc-83f4-e002066d0df9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.457237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113a5aa-2ab6-4dfc-83f4-e002066d0df9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.462029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.462653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.462925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.463190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.463875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:04:13.464195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.465834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.478732 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1167/2338] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:04:13.483752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd818d-8b1f-45fe-bbef-16fd9b23af2f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.493512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd818d-8b1f-45fe-bbef-16fd9b23af2f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.498622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.499223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.499500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.499771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.500448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.502391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.534486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1172/2339] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:04:13 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:04:13.540424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.551498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08776-65a0-46c5-910e-c8005b7eda05 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.615369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.615988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.616375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.616651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.617292 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.619410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.643698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1168/2340] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:04:13 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:04:13.648373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.659864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ae68-882e-436b-bc0e-3bc9fd797ebf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.665459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.665745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.666012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.666286 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.666934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.668597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.679344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1173/2341] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:04:13.683849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.694103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b9726-1a30-4d5e-8807-8de896081366 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.700801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e41d2-35f0-4d6c-a226-a323fbc9d110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.701405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e41d2-35f0-4d6c-a226-a323fbc9d110 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.701681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e41d2-35f0-4d6c-a226-a323fbc9d110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.701949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e41d2-35f0-4d6c-a226-a323fbc9d110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.702587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e41d2-35f0-4d6c-a226-a323fbc9d110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.704269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e41d2-35f0-4d6c-a226-a323fbc9d110 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.715811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1169/2342] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:04:13.719996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.733801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870733b-5a26-4f6f-875c-0ecc7df50a07 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.741268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.742230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.742608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.743009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] PATH_INFO: `/v3/projects/e6c32d7007a1428a8bde1b73e5883db6/users/9b3a59d52ce8432c964f8c94a6e96a4d/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.757817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c32d7007a1428a8bde1b73e5883db6, user_id=9b3a59d52ce8432c964f8c94a6e96a4d, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.760647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.775726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1174/2343] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:04:13 2022] PUT /identity/v3/projects/e6c32d7007a1428a8bde1b73e5883db6/users/9b3a59d52ce8432c964f8c94a6e96a4d/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:13.782510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb8c9e-eee8-4bbf-923d-ebe7bf9eecb0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.782872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb8c9e-eee8-4bbf-923d-ebe7bf9eecb0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.783239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb8c9e-eee8-4bbf-923d-ebe7bf9eecb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.804490 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-57bb8c9e-eee8-4bbf-923d-ebe7bf9eecb0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:13.805881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-57bb8c9e-eee8-4bbf-923d-ebe7bf9eecb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:13.874019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1170/2344] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:13.884053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:13.885897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:13.944724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042ccfe-ca6c-4b43-bef9-b0f159a484ee admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:13.948639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af8f36-493f-487e-a358-54752aef73d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:13.949272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af8f36-493f-487e-a358-54752aef73d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:13.949557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af8f36-493f-487e-a358-54752aef73d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:13.949828 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af8f36-493f-487e-a358-54752aef73d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:13.958952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af8f36-493f-487e-a358-54752aef73d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:13.960444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af8f36-493f-487e-a358-54752aef73d4 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:13.970506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1175/2345] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:17.446632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71944a-29de-43bc-a7fe-6e8f167a5846 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:17.447176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71944a-29de-43bc-a7fe-6e8f167a5846 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:17.447176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71944a-29de-43bc-a7fe-6e8f167a5846 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:17.448806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1171/2346] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:04:17.463771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:17.464832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:17.465433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:17.505340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] MFA Rules not processed for user `2b0d695705124ac0b5d0a63e70ad45ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:17.506576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:17.514121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:17.618636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1176/2347] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:04:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:17.642026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71944a-29de-43bc-a7fe-6e8f167a5846 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:17.644767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-df71944a-29de-43bc-a7fe-6e8f167a5846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:17.672138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71944a-29de-43bc-a7fe-6e8f167a5846 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:17.684971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:17.685701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:17.686435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:17.686905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:17.720905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:17.724444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:17.754325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1172/2348] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:19.010668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:19.013355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:19.037515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09caba85-1ded-4af5-94c0-65794fc8afdd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:19.043485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:19.048872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:19.049161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:19.049452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] PATH_INFO: `/v3/users/b0bd7cbe6cbe4019ba478b57c9ae94d4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:19.050843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0bd7cbe6cbe4019ba478b57c9ae94d4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:19.052933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:19.112100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0bd7cbe6cbe4019ba478b57c9ae94d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.121251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0bd7cbe6cbe4019ba478b57c9ae94d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.133752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0bd7cbe6cbe4019ba478b57c9ae94d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.134500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0bd7cbe6cbe4019ba478b57c9ae94d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.141898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0bd7cbe6cbe4019ba478b57c9ae94d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.153851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1177/2349] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:04:19 2022] DELETE /identity/v3/users/b0bd7cbe6cbe4019ba478b57c9ae94d4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:19.492764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:19.503480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36e7bd-b474-477f-9e48-1017e45890d4 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:19.550300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:19.581600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:19.582547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:19.582994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:19.583265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] PATH_INFO: `/v3/projects/b7d624d60c47492e8aeda03a32c6d84b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:19.585716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d624d60c47492e8aeda03a32c6d84b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:19.586717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:19.685643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d624d60c47492e8aeda03a32c6d84b', 'request_id': 'req-41e73b92-1d28-4175-b80b-5c8174657c3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.692110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d624d60c47492e8aeda03a32c6d84b', 'request_id': 'req-41e73b92-1d28-4175-b80b-5c8174657c3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.699124 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d624d60c47492e8aeda03a32c6d84b', 'request_id': 'req-41e73b92-1d28-4175-b80b-5c8174657c3e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.715162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] The token cache is being invalidate because project b7d624d60c47492e8aeda03a32c6d84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:04:19.715737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.716428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:19.736298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:19.738230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:19.742266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1173/2350] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:04:19 2022] DELETE /identity/v3/projects/b7d624d60c47492e8aeda03a32c6d84b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:19.766583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762da93-4170-4171-9874-5ba80cff11c5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:19.885800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:19.886753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:19.902956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:19.906530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] PATH_INFO: `/v3/users/36c7111e5b0c45609cc615258c8a30ea` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:19.910831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=36c7111e5b0c45609cc615258c8a30ea)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:19.915165 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:19.966810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:20.004761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36c7111e5b0c45609cc615258c8a30ea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.029913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36c7111e5b0c45609cc615258c8a30ea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.046011 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36c7111e5b0c45609cc615258c8a30ea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.047689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36c7111e5b0c45609cc615258c8a30ea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.058199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-974da3a1-8fda-4cf0-becd-43d6ed32a3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36c7111e5b0c45609cc615258c8a30ea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.069013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1178/2351] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:04:19 2022] DELETE /identity/v3/users/36c7111e5b0c45609cc615258c8a30ea => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:20.326888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:20.328416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:20.349044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e73b92-1d28-4175-b80b-5c8174657c3e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:20.428207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:20.428871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:20.429145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:20.429413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] PATH_INFO: `/v3/projects/fd657dea7ffa46138601eec429d624f3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:20.430953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd657dea7ffa46138601eec429d624f3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:20.433445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:20.494461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd657dea7ffa46138601eec429d624f3', 'request_id': 'req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.502489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd657dea7ffa46138601eec429d624f3', 'request_id': 'req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.510997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd657dea7ffa46138601eec429d624f3', 'request_id': 'req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.529755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:20.536676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] The token cache is being invalidate because project fd657dea7ffa46138601eec429d624f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:04:20.536676 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.537413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5fd7ff22-47e8-4d35-a068-4fe4d3ae76f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:20.540943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1174/2352] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:04:20 2022] DELETE /identity/v3/projects/fd657dea7ffa46138601eec429d624f3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:20.546245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dd85-c429-4294-9fd8-5f3b208be3ba None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:20.546619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dd85-c429-4294-9fd8-5f3b208be3ba None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:20.547039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860dd85-c429-4294-9fd8-5f3b208be3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:20.570786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d860dd85-c429-4294-9fd8-5f3b208be3ba None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:20.572694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d860dd85-c429-4294-9fd8-5f3b208be3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:20.662924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1179/2353] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:20.671389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d7f1-3340-488d-849a-d0ae7c1caf3d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:20.671913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d7f1-3340-488d-849a-d0ae7c1caf3d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:20.672771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283d7f1-3340-488d-849a-d0ae7c1caf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:20.704278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e283d7f1-3340-488d-849a-d0ae7c1caf3d None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:20.706046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e283d7f1-3340-488d-849a-d0ae7c1caf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:20.729052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860dd85-c429-4294-9fd8-5f3b208be3ba None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:20.737795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1175/2354] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:20.740475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860dd85-c429-4294-9fd8-5f3b208be3ba None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:20.748771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:20.749574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:20.749994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:20.750623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:20.751121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:04:20.751546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:20.753531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:20.772587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1180/2355] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:04:20.777127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d7f1-3340-488d-849a-d0ae7c1caf3d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:20.788551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283d7f1-3340-488d-849a-d0ae7c1caf3d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:20.794053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:20.794677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:20.794955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:20.795230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:20.796917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:20.798857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:20.863370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:20.879063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f852f-a7cf-4f86-bb7f-e5e43ecfc4ed admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:20.879649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1176/2356] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:04:20 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:04:21.046810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:21.048078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:21.048560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:21.048994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:21.049808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:21.057921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:21.148450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1181/2357] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:04:20 2022] POST /identity/v3/users => generated 286 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:04:21.152951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:21.170147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7936-ed71-4e57-90e8-6f75bc304e0c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:21.190197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:21.190852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:21.191133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:21.191405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:21.192077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:21.194157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:21.225073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:21.225444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1177/2358] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:04:21.237494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae109f5e-2b63-444c-a7d3-11f06ff26cae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:21.243086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79061d2d-cabc-46f2-84ab-fef5e9f636d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:21.243820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79061d2d-cabc-46f2-84ab-fef5e9f636d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:21.244113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79061d2d-cabc-46f2-84ab-fef5e9f636d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:21.244702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79061d2d-cabc-46f2-84ab-fef5e9f636d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:21.245437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79061d2d-cabc-46f2-84ab-fef5e9f636d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:21.248630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79061d2d-cabc-46f2-84ab-fef5e9f636d0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:21.269505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:21.275369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1182/2359] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:04:21.303364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac97b4b-19f1-4083-a7f0-a61c3aa05bd3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:21.326061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:21.326766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:21.327049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:21.327321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] PATH_INFO: `/v3/projects/2d6228f311304b7b8e6354d865e16e9c/users/816427b5539240ad9aa22150a922bdc0/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:21.360115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6228f311304b7b8e6354d865e16e9c, user_id=816427b5539240ad9aa22150a922bdc0, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:21.362822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:21.403790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1178/2360] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:04:21 2022] PUT /identity/v3/projects/2d6228f311304b7b8e6354d865e16e9c/users/816427b5539240ad9aa22150a922bdc0/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:04:21.409540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2a2d5-a6b7-4ef9-9548-d49a432e8345 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:21.409831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2a2d5-a6b7-4ef9-9548-d49a432e8345 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:21.410113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2a2d5-a6b7-4ef9-9548-d49a432e8345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:21.424530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:21.444734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-44b2a2d5-a6b7-4ef9-9548-d49a432e8345 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:21.446607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44b2a2d5-a6b7-4ef9-9548-d49a432e8345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:21.568362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1183/2361] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:21.576634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:21.578414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:21.659385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88844714-fc2b-41ba-9939-6353a2244435 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:21.664078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:21.665046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:21.665817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:21.666030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:21.676908 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:21.678712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:21.692275 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1179/2362] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:22.366741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0702bf-2973-405c-a8e1-d8f9a4922f39 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:22.367919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0702bf-2973-405c-a8e1-d8f9a4922f39 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:22.368454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0702bf-2973-405c-a8e1-d8f9a4922f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:22.418160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9f0702bf-2973-405c-a8e1-d8f9a4922f39 None None] MFA Rules not processed for user `9b3a59d52ce8432c964f8c94a6e96a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:22.420463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9f0702bf-2973-405c-a8e1-d8f9a4922f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:22.518476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1184/2363] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:22.523192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:22.524978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:22.540434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb694f-8447-4b36-ac9e-5548a0318c5c service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:22.545185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:22.545803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:22.546081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:22.546741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:22.547093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:22.549230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:22.583704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1180/2364] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:04:22 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:04:22.587881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0702bf-2973-405c-a8e1-d8f9a4922f39 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:22.605049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0702bf-2973-405c-a8e1-d8f9a4922f39 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:22.674278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:22.674908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:22.675333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:22.675559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:22.676099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:22.678308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:22.705966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1185/2365] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:04:22 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:04:22.707505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:22.724613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406dcc70-bfa1-4215-9581-214e7c876914 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:22.733733 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:22.734403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:22.734683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:22.734955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:22.735654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:22.737422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:22.751010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1181/2366] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:04:22.755807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:22.769653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e5e40-02e6-4bcd-86cc-7bf3d345a405 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:22.777119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:22.777864 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:22.778134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:22.778412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] PATH_INFO: `/v3/projects/c78ebda5da1441abbadbca3820920988/users/ac154588bdd64c58b43c50c7e78e4d14/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:22.795383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78ebda5da1441abbadbca3820920988, user_id=ac154588bdd64c58b43c50c7e78e4d14, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:22.798113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:22.810306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:22.816545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1186/2367] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:04:22 2022] PUT /identity/v3/projects/c78ebda5da1441abbadbca3820920988/users/ac154588bdd64c58b43c50c7e78e4d14/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:22.820277 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:22.831584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cea99-2ce6-4a26-a003-314baed5a87a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:22.915041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929f67a-5aef-448c-8122-c533210dc8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:22.915926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929f67a-5aef-448c-8122-c533210dc8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:22.916380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929f67a-5aef-448c-8122-c533210dc8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:22.916664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929f67a-5aef-448c-8122-c533210dc8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:22.917642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929f67a-5aef-448c-8122-c533210dc8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:22.919278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929f67a-5aef-448c-8122-c533210dc8f6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:22.939228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:22.950600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1182/2368] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:04:22.951202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:22.968763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5655d83-a386-4354-9401-8d4f77b6e5f9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:22.981531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:22.982360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:22.982710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:22.983054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] PATH_INFO: `/v3/projects/c78ebda5da1441abbadbca3820920988/users/ac154588bdd64c58b43c50c7e78e4d14/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:22.988706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78ebda5da1441abbadbca3820920988, user_id=ac154588bdd64c58b43c50c7e78e4d14, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:22.995926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:23.018123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1187/2369] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:04:22 2022] PUT /identity/v3/projects/c78ebda5da1441abbadbca3820920988/users/ac154588bdd64c58b43c50c7e78e4d14/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:23.338037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26de282-1a9b-4d6b-bc0b-ac056ee72d04 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:23.338555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26de282-1a9b-4d6b-bc0b-ac056ee72d04 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:23.339019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26de282-1a9b-4d6b-bc0b-ac056ee72d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:23.377461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c26de282-1a9b-4d6b-bc0b-ac056ee72d04 None None] MFA Rules not processed for user `816427b5539240ad9aa22150a922bdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:23.378946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c26de282-1a9b-4d6b-bc0b-ac056ee72d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:23.737252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1183/2370] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:23.747416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:23.751134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:23.936986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667943d-f535-4325-b11e-b237d7cff3e8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:23.946733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c1941-379f-4f77-a9c1-a9c50f192e2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:23.948335 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1941-379f-4f77-a9c1-a9c50f192e2c service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:23.948777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1941-379f-4f77-a9c1-a9c50f192e2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:23.949197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c1941-379f-4f77-a9c1-a9c50f192e2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:23.961300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1941-379f-4f77-a9c1-a9c50f192e2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:23.963417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c1941-379f-4f77-a9c1-a9c50f192e2c service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:23.981428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1188/2371] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:26.622394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:26.622923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:26.622923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:26.657891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] MFA Rules not processed for user `816427b5539240ad9aa22150a922bdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:26.659259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:26.667746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1184/2372] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:29.457012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:29.457512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:29.457609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:29.490048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] MFA Rules not processed for user `ac154588bdd64c58b43c50c7e78e4d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:29.491205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:29.618390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1189/2373] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:04:29.633498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:29.635382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:29.758555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fded07d-b64e-40a7-bef9-095466d0fd07 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:29.765484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7e4ac2-27b0-43ab-bf6d-786ba398f328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:29.768695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e4ac2-27b0-43ab-bf6d-786ba398f328 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:29.769100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e4ac2-27b0-43ab-bf6d-786ba398f328 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:29.769438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7e4ac2-27b0-43ab-bf6d-786ba398f328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:29.913314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e4ac2-27b0-43ab-bf6d-786ba398f328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:29.915031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7e4ac2-27b0-43ab-bf6d-786ba398f328 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:29.949543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1185/2374] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:39.613651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:39.925607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:48.600654 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:48.602655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:48.687831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42955c-6fa1-4277-ac64-a1e7f149aa0c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:48.692636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:48.693317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:48.693950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:48.694230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] PATH_INFO: `/v3/users/91840d1a9c674699b3c18efa664b28dd` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:48.695389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91840d1a9c674699b3c18efa664b28dd)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:48.697376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:48.758463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91840d1a9c674699b3c18efa664b28dd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:48.771385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91840d1a9c674699b3c18efa664b28dd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:48.787422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91840d1a9c674699b3c18efa664b28dd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:48.788220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91840d1a9c674699b3c18efa664b28dd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:48.802527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91840d1a9c674699b3c18efa664b28dd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:48.814566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1190/2375] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:04:48 2022] DELETE /identity/v3/users/91840d1a9c674699b3c18efa664b28dd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:48.816048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:48.816561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:48.816660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:48.849336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:48.850583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:48.916847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1186/2376] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:04:48.926987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:48.928592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:49.017093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f8145-680e-469d-978e-8486ce3000d9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:49.022689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:49.022689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:49.022973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:49.023158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:49.034058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:49.035523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:49.045963 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1191/2377] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:49.346136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:49.347719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:49.362719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca305c-fb0b-414b-b0ed-494910aee68f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:49.368862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:49.369513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:49.369943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:49.370057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] PATH_INFO: `/v3/projects/0bd6710c4fc140509913dd76d5511dbb` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:49.371221 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd6710c4fc140509913dd76d5511dbb)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:49.373181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:49.426964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd6710c4fc140509913dd76d5511dbb', 'request_id': 'req-9abdde5d-b533-406a-b0ac-813be2ff4e53'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:49.432724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd6710c4fc140509913dd76d5511dbb', 'request_id': 'req-9abdde5d-b533-406a-b0ac-813be2ff4e53'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:49.439356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd6710c4fc140509913dd76d5511dbb', 'request_id': 'req-9abdde5d-b533-406a-b0ac-813be2ff4e53'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:49.454209 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] The token cache is being invalidate because project 0bd6710c4fc140509913dd76d5511dbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:04:49.454494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:49.455097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9abdde5d-b533-406a-b0ac-813be2ff4e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:04:49.456645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1187/2378] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:04:49 2022] DELETE /identity/v3/projects/0bd6710c4fc140509913dd76d5511dbb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:04:52.138284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:04:52.139821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:52.167199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:04:52.213712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af549e-0e2e-400e-a96b-6c522cfc87d3 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:04:52.280115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c5e4f-e548-4144-ad84-0c4e0ad6a9df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:04:52.280830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c5e4f-e548-4144-ad84-0c4e0ad6a9df service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:52.281910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c5e4f-e548-4144-ad84-0c4e0ad6a9df service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:52.282041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c5e4f-e548-4144-ad84-0c4e0ad6a9df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:52.284174 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b93c5e4f-e548-4144-ad84-0c4e0ad6a9df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:52.298374 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c5e4f-e548-4144-ad84-0c4e0ad6a9df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:04:52.301598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c5e4f-e548-4144-ad84-0c4e0ad6a9df service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:04:52.310969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1192/2379] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:04:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:04:53.719191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e560c5-fa2b-4c70-ab34-e647e3c09d87 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:04:53.719775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e560c5-fa2b-4c70-ab34-e647e3c09d87 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:04:53.719775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e560c5-fa2b-4c70-ab34-e647e3c09d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:04:53.740122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b4e560c5-fa2b-4c70-ab34-e647e3c09d87 None None] MFA Rules not processed for user `816427b5539240ad9aa22150a922bdc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:04:53.741342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b4e560c5-fa2b-4c70-ab34-e647e3c09d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:04:53.797217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1188/2380] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:04:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:02.307470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=48 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:05:03.788844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:05:09.448604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:09.449257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:09.449257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:09.472413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] MFA Rules not processed for user `9b3a59d52ce8432c964f8c94a6e96a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:09.473621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:09.532522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1193/2381] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:05:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:14.044556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8817d6-59b3-40e0-bd8d-89982e841d83 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:14.045165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8817d6-59b3-40e0-bd8d-89982e841d83 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:14.045309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8817d6-59b3-40e0-bd8d-89982e841d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:14.070044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2f8817d6-59b3-40e0-bd8d-89982e841d83 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:14.071555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2f8817d6-59b3-40e0-bd8d-89982e841d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:14.176048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1189/2382] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:05:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:14.229071 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:14.230618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:14.336905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ba4e1-e903-4983-b502-a80392fcd5e6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:14.341800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:14.342417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:14.342697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:14.342972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:14.351858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:14.353571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:14.361933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1194/2383] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:05:24.148497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=25 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:05:24.358809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=57 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:05:31.167927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308430c-e0a1-4538-ad7c-4d055499fb02 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:31.168522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308430c-e0a1-4538-ad7c-4d055499fb02 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:31.168522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308430c-e0a1-4538-ad7c-4d055499fb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:31.189965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3308430c-e0a1-4538-ad7c-4d055499fb02 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:31.191185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3308430c-e0a1-4538-ad7c-4d055499fb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:31.198852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1190/2384] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:05:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:31.211420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:31.223307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b02b77-f299-4ba8-a342-856639006c82 service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:31.229969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94085d6a-6413-4fc1-9ac6-dc2edf6f441e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:31.230702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085d6a-6413-4fc1-9ac6-dc2edf6f441e service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:31.231083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085d6a-6413-4fc1-9ac6-dc2edf6f441e service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:31.231412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94085d6a-6413-4fc1-9ac6-dc2edf6f441e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:31.240291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94085d6a-6413-4fc1-9ac6-dc2edf6f441e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:31.241989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94085d6a-6413-4fc1-9ac6-dc2edf6f441e service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:31.250831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1195/2385] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:05:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:05:39.507039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ec8e-241d-474e-9403-ae3814ebd104 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:39.507743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ec8e-241d-474e-9403-ae3814ebd104 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:39.507743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742ec8e-241d-474e-9403-ae3814ebd104 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:39.510441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1191/2386] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 05 17:05:39.515331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c911-ce89-46c9-b576-23c39a37cf1f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:39.515869 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c911-ce89-46c9-b576-23c39a37cf1f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:39.516031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8c911-ce89-46c9-b576-23c39a37cf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:39.540047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5cf8c911-ce89-46c9-b576-23c39a37cf1f None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:39.541833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5cf8c911-ce89-46c9-b576-23c39a37cf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:39.622238 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1196/2387] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:05:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:39.636273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742ec8e-241d-474e-9403-ae3814ebd104 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:39.638373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8742ec8e-241d-474e-9403-ae3814ebd104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:39.772887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742ec8e-241d-474e-9403-ae3814ebd104 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:39.782198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2fd61-ee7d-4ac7-8590-1feb88752489 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05a016553950460dbb736ced2cae4f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:39.782851 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2fd61-ee7d-4ac7-8590-1feb88752489 service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:39.783282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2fd61-ee7d-4ac7-8590-1feb88752489 service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:39.783711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2fd61-ee7d-4ac7-8590-1feb88752489 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:39.799382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2fd61-ee7d-4ac7-8590-1feb88752489 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:39.801137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2fd61-ee7d-4ac7-8590-1feb88752489 service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:39.821059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1192/2388] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:05:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:05:39.917314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561014e4-c258-4790-9ee4-7604664cee1a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:39.917729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561014e4-c258-4790-9ee4-7604664cee1a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:39.918088 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561014e4-c258-4790-9ee4-7604664cee1a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:39.919392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1197/2389] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:05:39.925136 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ea345-294d-4a3e-bd4f-09c333d2b41d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:39.925430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ea345-294d-4a3e-bd4f-09c333d2b41d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:39.925707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ea345-294d-4a3e-bd4f-09c333d2b41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:39.946179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5d9ea345-294d-4a3e-bd4f-09c333d2b41d None None] MFA Rules not processed for user `fcbcd7638cc84cdd87a1eb98f5926425`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:39.947445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5d9ea345-294d-4a3e-bd4f-09c333d2b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:40.008127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1193/2390] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:05:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:05:40.021132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561014e4-c258-4790-9ee4-7604664cee1a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:40.023312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-561014e4-c258-4790-9ee4-7604664cee1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:40.044321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561014e4-c258-4790-9ee4-7604664cee1a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:40.048484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b04c5d-8d4e-4bfb-9638-c3cfae29fa07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:40.049282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04c5d-8d4e-4bfb-9638-c3cfae29fa07 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:40.049717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04c5d-8d4e-4bfb-9638-c3cfae29fa07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:40.050138 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b04c5d-8d4e-4bfb-9638-c3cfae29fa07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:40.059065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b04c5d-8d4e-4bfb-9638-c3cfae29fa07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:40.060656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b04c5d-8d4e-4bfb-9638-c3cfae29fa07 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:40.070905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1198/2391] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:05:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:05:40.169122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1c279-9c0d-4eec-a1cb-83e78ec897f1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:40.169715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1c279-9c0d-4eec-a1cb-83e78ec897f1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:40.170007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1c279-9c0d-4eec-a1cb-83e78ec897f1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:40.171470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1194/2392] 10.222.0.23 () {54 vars in 921 bytes} [Mon Sep 5 17:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 05 17:05:40.178474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956bd88-63a6-4e60-96b3-58d6a24db3f3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:40.178767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956bd88-63a6-4e60-96b3-58d6a24db3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:40.179041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956bd88-63a6-4e60-96b3-58d6a24db3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:40.229412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6956bd88-63a6-4e60-96b3-58d6a24db3f3 None None] MFA Rules not processed for user `c30e20ce4d63452095aedb60249f0cce`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:40.232728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6956bd88-63a6-4e60-96b3-58d6a24db3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:40.260944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1199/2393] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Sep 5 17:05:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:05:40.272134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1c279-9c0d-4eec-a1cb-83e78ec897f1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:40.289922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1c279-9c0d-4eec-a1cb-83e78ec897f1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:40.295343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:40.296441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:40.296959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:40.297582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:40.324639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:40.326687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:40.360834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1195/2394] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:05:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:05:47.729187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478999-0a29-4460-ae83-34ff44d85c1b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:47.729776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478999-0a29-4460-ae83-34ff44d85c1b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:47.729902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478999-0a29-4460-ae83-34ff44d85c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:47.749013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-75478999-0a29-4460-ae83-34ff44d85c1b None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:47.750214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-75478999-0a29-4460-ae83-34ff44d85c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:47.757416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1200/2395] 10.222.0.23 () {58 vars in 967 bytes} [Mon Sep 5 17:05:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:47.770538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:47.782976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118aae91-3445-4b3e-9fb6-cc56061f55ed service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:47.788537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:47.789149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:47.789431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:47.789726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:47.799413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:47.801067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:47.810368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1196/2396] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:05:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:05:48.178189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:48.178863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:48.179302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:48.200593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:48.201892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:48.210103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1201/2397] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:05:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:48.227469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:48.238975 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f71cfd-4cc0-4377-82e1-638ee4cc6129 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:48.245854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:48.247620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:48.247980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:48.248247 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:48.264402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:48.266366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:48.275844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1197/2398] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:05:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:05:50.233803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:50.236499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:50.425201 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c7c371-9ca1-49f3-9d6d-e67b62fec4c7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:50.431881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:50.433615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:50.434008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:50.434535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] PATH_INFO: `/v3/users/816427b5539240ad9aa22150a922bdc0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:50.435897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=816427b5539240ad9aa22150a922bdc0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:50.438017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:50.547410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816427b5539240ad9aa22150a922bdc0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:50.550618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:05:50.569951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816427b5539240ad9aa22150a922bdc0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:50.586973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816427b5539240ad9aa22150a922bdc0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:50.587794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816427b5539240ad9aa22150a922bdc0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:50.600797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bb55725a-d92f-482b-9807-10b9473a9fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816427b5539240ad9aa22150a922bdc0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:50.614592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1202/2399] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:05:50 2022] DELETE /identity/v3/users/816427b5539240ad9aa22150a922bdc0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:05:50.824686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:50.836804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe38ad-cce5-4ca9-b924-640d47649675 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:50.939508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:50.940344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:50.940904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:50.941291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] PATH_INFO: `/v3/projects/2d6228f311304b7b8e6354d865e16e9c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:50.942798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6228f311304b7b8e6354d865e16e9c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:50.945024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:50.959750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:05:51.045346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6228f311304b7b8e6354d865e16e9c', 'request_id': 'req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:51.072345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6228f311304b7b8e6354d865e16e9c', 'request_id': 'req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:51.080781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6228f311304b7b8e6354d865e16e9c', 'request_id': 'req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:51.108316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] The token cache is being invalidate because project 2d6228f311304b7b8e6354d865e16e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:05:51.109066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:51.110103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d1486b3d-374b-436c-9f4d-0e6e6a0cce29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:05:51.112257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1198/2400] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:05:50 2022] DELETE /identity/v3/projects/2d6228f311304b7b8e6354d865e16e9c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:05:51.117021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829788eb-3914-48b6-a517-e412e7be64a7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.117310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829788eb-3914-48b6-a517-e412e7be64a7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.117590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829788eb-3914-48b6-a517-e412e7be64a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.143862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-829788eb-3914-48b6-a517-e412e7be64a7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:51.145608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-829788eb-3914-48b6-a517-e412e7be64a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:51.237361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1203/2401] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:51.238700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b433ca-572d-46db-a1db-88bc95db2d3e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.239008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b433ca-572d-46db-a1db-88bc95db2d3e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.239288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b433ca-572d-46db-a1db-88bc95db2d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.262313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-37b433ca-572d-46db-a1db-88bc95db2d3e None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:51.263746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37b433ca-572d-46db-a1db-88bc95db2d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:51.273504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1199/2402] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:05:51.278790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829788eb-3914-48b6-a517-e412e7be64a7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:51.289497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829788eb-3914-48b6-a517-e412e7be64a7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:51.294340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:51.294976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.295251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.295523 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.296404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:05:51.296711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:51.298694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:51.312221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1204/2403] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:05:51.316525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b433ca-572d-46db-a1db-88bc95db2d3e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:51.327523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b433ca-572d-46db-a1db-88bc95db2d3e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:51.332196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:51.332815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.333095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.333363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.334021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:51.335958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:51.380114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1200/2404] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:05:51 2022] POST /identity/v3/projects => generated 399 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:05:51.381674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:51.392803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c486035-669a-4024-94b7-57339e0de81a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:51.463886 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:51.464708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.464992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.465440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.466098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:51.468452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:51.495128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1205/2405] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:05:51 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:05:51.500061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:51.512489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e186f6-9bb3-4b53-bf1f-179288db2c5e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:51.518147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:51.518723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.519007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.519239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.519888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:51.521735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:51.535755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1201/2406] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:05:51.539717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:51.551230 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ac0b0-867c-44be-a2ba-6762022bb16a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:51.556068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a79e0e-c3ab-4279-a4a3-f7bb5860a5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:51.556727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a79e0e-c3ab-4279-a4a3-f7bb5860a5ae admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.557010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a79e0e-c3ab-4279-a4a3-f7bb5860a5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.557282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a79e0e-c3ab-4279-a4a3-f7bb5860a5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.558037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a79e0e-c3ab-4279-a4a3-f7bb5860a5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:51.559867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a79e0e-c3ab-4279-a4a3-f7bb5860a5ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:51.580164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1206/2407] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:05:51.584441 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:51.599240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0698b-f744-46d8-bb8b-f7394221d573 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:51.607723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:51.608900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.609401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.609834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] PATH_INFO: `/v3/projects/e022d18dd8904b9292dcb8c10f857293/users/e22224666bf14d22a539148de00a0e49/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.634384 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e022d18dd8904b9292dcb8c10f857293, user_id=e22224666bf14d22a539148de00a0e49, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:51.637859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:51.660003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1202/2408] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:05:51 2022] PUT /identity/v3/projects/e022d18dd8904b9292dcb8c10f857293/users/e22224666bf14d22a539148de00a0e49/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:05:51.672267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018715-b565-4b79-b2d6-8d38c53950f0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.672641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018715-b565-4b79-b2d6-8d38c53950f0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.672831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018715-b565-4b79-b2d6-8d38c53950f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.703878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-88018715-b565-4b79-b2d6-8d38c53950f0 None None] MFA Rules not processed for user `e22224666bf14d22a539148de00a0e49`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:05:51.705156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-88018715-b565-4b79-b2d6-8d38c53950f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:51.725086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:05:51.788081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1207/2409] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:05:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:05:51.800504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:05:51.802461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:05:51.865755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a72f5a-5df1-4552-88d9-060df8c5ee9a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:05:51.869400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443b645-8666-40eb-b8f0-fc496d3b0d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:05:51.870049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443b645-8666-40eb-b8f0-fc496d3b0d30 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:05:51.870571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443b645-8666-40eb-b8f0-fc496d3b0d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:05:51.871373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443b645-8666-40eb-b8f0-fc496d3b0d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:05:51.880663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443b645-8666-40eb-b8f0-fc496d3b0d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:05:51.882284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443b645-8666-40eb-b8f0-fc496d3b0d30 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:05:51.893532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1203/2410] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:05:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:06:01.790119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=18 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:01.888555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:05.408950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88018715-b565-4b79-b2d6-8d38c53950f0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:05.483112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88018715-b565-4b79-b2d6-8d38c53950f0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:05.487976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:05.488660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:05.488938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:05.489229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] PATH_INFO: `/v3/users/e22224666bf14d22a539148de00a0e49` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:05.491124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e22224666bf14d22a539148de00a0e49)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:05.493145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:05.544749 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e22224666bf14d22a539148de00a0e49'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:05.555120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e22224666bf14d22a539148de00a0e49'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:05.567774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e22224666bf14d22a539148de00a0e49'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:05.568483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e22224666bf14d22a539148de00a0e49'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:05.575447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e22224666bf14d22a539148de00a0e49'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:05.581483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1208/2411] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:06:05 2022] DELETE /identity/v3/users/e22224666bf14d22a539148de00a0e49 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:06:05.586167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:05.586659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:05.586778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:05.607289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:05.608518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:05.685364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1204/2412] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:05.695661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:05.697160 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:05.769931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb64f3-4528-4833-bbd1-46a846104d60 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:05.774525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da057edb-23f7-472b-b30f-5337ceaff9dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:05.775139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057edb-23f7-472b-b30f-5337ceaff9dc service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:05.775411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057edb-23f7-472b-b30f-5337ceaff9dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:05.775701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da057edb-23f7-472b-b30f-5337ceaff9dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:05.786116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da057edb-23f7-472b-b30f-5337ceaff9dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:05.787824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da057edb-23f7-472b-b30f-5337ceaff9dc service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:05.798151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1209/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:06:06.203675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.205196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:06.219265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2381-9422-438b-846c-79f985a786c2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.233623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.234297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.234572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.234843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] PATH_INFO: `/v3/projects/e022d18dd8904b9292dcb8c10f857293` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.236315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e022d18dd8904b9292dcb8c10f857293)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.238735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.291868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e022d18dd8904b9292dcb8c10f857293', 'request_id': 'req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:06.297791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e022d18dd8904b9292dcb8c10f857293', 'request_id': 'req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:06.304503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e022d18dd8904b9292dcb8c10f857293', 'request_id': 'req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:06.318675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] The token cache is being invalidate because project e022d18dd8904b9292dcb8c10f857293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:06:06.318963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:06.319583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:06:06.321164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1205/2414] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:06:06 2022] DELETE /identity/v3/projects/e022d18dd8904b9292dcb8c10f857293 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:06:06.326705 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10fdee-0be2-4020-ba82-75df6a512b78 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.327077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10fdee-0be2-4020-ba82-75df6a512b78 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.327442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10fdee-0be2-4020-ba82-75df6a512b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.348024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5d10fdee-0be2-4020-ba82-75df6a512b78 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:06.349346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5d10fdee-0be2-4020-ba82-75df6a512b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:06.364457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.366243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:06.371745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:06.424944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1210/2415] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:06.437021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71056461-0bcd-4539-96eb-1949aab26beb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.438996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71056461-0bcd-4539-96eb-1949aab26beb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.439275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71056461-0bcd-4539-96eb-1949aab26beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.439965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c5cc8-24e1-40be-aa0e-89907fd8c853 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.445570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.446143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.446426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.446699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.449318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:06.483806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-71056461-0bcd-4539-96eb-1949aab26beb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:06.485154 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-71056461-0bcd-4539-96eb-1949aab26beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:06.496494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1211/2416] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:06.508140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71056461-0bcd-4539-96eb-1949aab26beb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.526790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71056461-0bcd-4539-96eb-1949aab26beb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.536608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.536608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.536608 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.538415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.539893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:06:06.540172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.542637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.548593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:06.565771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1212/2417] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:06:06.571809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.583382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3fddb-89db-4726-9b01-6af8db7a1447 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.586201 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.587664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.588840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.589561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.589943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.590333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.591125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.593192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.597708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1206/2418] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:06:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:06:06.628293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1213/2419] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:06:06 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:06:06.632742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.647278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e5430-80b9-4796-99a2-eb7eee28fbd6 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.714513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.715320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.715662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.716037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.717896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.719914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.746740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1207/2420] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:06:06 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:06:06.750867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.762542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8aa0b-c81d-4816-ad12-cc6594e435b9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.767734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.768451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.768728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.769099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.769768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.771882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.786365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1214/2421] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:06:06.791732 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.802928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa844b-c96f-4f8f-a4fd-427739c11c98 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.808234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c4d00-a2e7-4399-8c91-766f2dfef423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.808893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c4d00-a2e7-4399-8c91-766f2dfef423 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.809248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c4d00-a2e7-4399-8c91-766f2dfef423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.809589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c4d00-a2e7-4399-8c91-766f2dfef423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.810381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c4d00-a2e7-4399-8c91-766f2dfef423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.812173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c4d00-a2e7-4399-8c91-766f2dfef423 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.824015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1208/2422] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:06:06.829277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.839636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81436e8a-73a9-4cdd-8e20-9330ad330965 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:06.844310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:06.844980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.845268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.845551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] PATH_INFO: `/v3/projects/0b2e9de2e0a64e67beb01dc673c653e5/users/c8b8aa2ae43a4d89ad579a5fdd87d564/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.860960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2e9de2e0a64e67beb01dc673c653e5, user_id=c8b8aa2ae43a4d89ad579a5fdd87d564, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:06.863669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:06.881938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1215/2423] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:06:06 2022] PUT /identity/v3/projects/0b2e9de2e0a64e67beb01dc673c653e5/users/c8b8aa2ae43a4d89ad579a5fdd87d564/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:06:06.887046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e97b8-2502-4fbc-b809-f0b3be0083aa None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:06.887336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e97b8-2502-4fbc-b809-f0b3be0083aa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:06.887612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e97b8-2502-4fbc-b809-f0b3be0083aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:06.907731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-811e97b8-2502-4fbc-b809-f0b3be0083aa None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:06.908996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-811e97b8-2502-4fbc-b809-f0b3be0083aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:06.979044 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1209/2424] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:06.995503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:06.997465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:07.066431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b41672-41db-4894-a14d-91d0e49d15bd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:07.070136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:07.070767 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:07.071040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:07.071309 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:07.080452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:07.081903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:07.091688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1216/2425] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:06:11.454541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a86f8-27f2-4303-81a1-afbb6efecdec None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:11.455086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a86f8-27f2-4303-81a1-afbb6efecdec None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:11.455086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a86f8-27f2-4303-81a1-afbb6efecdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:11.491584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-590a86f8-27f2-4303-81a1-afbb6efecdec None None] MFA Rules not processed for user `c8b8aa2ae43a4d89ad579a5fdd87d564`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:11.493066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-590a86f8-27f2-4303-81a1-afbb6efecdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:11.540391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:11.568846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1210/2426] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:11.570208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:11.581450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d90115-ebff-4cd5-9820-fd132acab071 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:11.586810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:11.587435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:11.587715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:11.587984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:11.588662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:11.590625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:11.621203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1217/2427] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:06:11 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:06:11.625582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a86f8-27f2-4303-81a1-afbb6efecdec None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:11.637051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a86f8-27f2-4303-81a1-afbb6efecdec None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:11.701027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:11.701644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:11.701914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:11.702178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:11.702823 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:11.704809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:11.730628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1211/2428] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:06:11 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:06:11.734817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:11.746287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59267f61-e86b-4e36-97d7-3f3eb1baffc5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:11.752520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:11.753063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:11.753346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:11.753622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:11.755317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:11.755989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:11.769498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1218/2429] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:06:11.773866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:11.783938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7778c-8e25-4b10-935d-97c757c72f45 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:11.788827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:11.789482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:11.789756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:11.790026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] PATH_INFO: `/v3/projects/f8ff0444abda46338ef8c0c7b1f696cf/users/9e690642cf4947eba950ddead258617f/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:11.805313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ff0444abda46338ef8c0c7b1f696cf, user_id=9e690642cf4947eba950ddead258617f, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:11.807918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:11.824488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1212/2430] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:06:11 2022] PUT /identity/v3/projects/f8ff0444abda46338ef8c0c7b1f696cf/users/9e690642cf4947eba950ddead258617f/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:06:11.828692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:11.840370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcf105-87a6-418d-988c-71f38a0903ee admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:11.854183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:11.904178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ae0b10-9960-46a6-a1d1-0d5815be9f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:11.904832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae0b10-9960-46a6-a1d1-0d5815be9f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:11.905115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae0b10-9960-46a6-a1d1-0d5815be9f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:11.905391 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ae0b10-9960-46a6-a1d1-0d5815be9f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:11.906031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae0b10-9960-46a6-a1d1-0d5815be9f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:11.907674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ae0b10-9960-46a6-a1d1-0d5815be9f93 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:11.918002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1219/2431] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:06:11.922402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:11.933735 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8c3bd-3db5-463d-aefa-907bc3bec4fb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:11.939376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:11.940045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:11.940358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:11.940628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] PATH_INFO: `/v3/projects/f8ff0444abda46338ef8c0c7b1f696cf/users/9e690642cf4947eba950ddead258617f/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:11.942930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ff0444abda46338ef8c0c7b1f696cf, user_id=9e690642cf4947eba950ddead258617f, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:11.945741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:11.959039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1213/2432] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:06:11 2022] PUT /identity/v3/projects/f8ff0444abda46338ef8c0c7b1f696cf/users/9e690642cf4947eba950ddead258617f/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:06:16.929499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041236b-38aa-481e-a8a2-9f567236c06a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:16.929971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041236b-38aa-481e-a8a2-9f567236c06a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:16.930190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041236b-38aa-481e-a8a2-9f567236c06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:16.988937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e041236b-38aa-481e-a8a2-9f567236c06a None None] MFA Rules not processed for user `9e690642cf4947eba950ddead258617f`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:16.990290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e041236b-38aa-481e-a8a2-9f567236c06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:17.070405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1220/2433] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:17.085510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:17.087319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:17.147606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aedc9-b22b-4087-a46f-fb9207b39f8f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:17.151353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05a016553950460dbb736ced2cae4f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:17.151974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:17.152338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:17.152615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:17.204525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:17.216334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:17.217896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:17.228084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1214/2434] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:06:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:06:17.392404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f366f-99f3-4377-8e29-7d7b3439f061 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:17.392790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f366f-99f3-4377-8e29-7d7b3439f061 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:17.395232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f366f-99f3-4377-8e29-7d7b3439f061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:17.416255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c93f366f-99f3-4377-8e29-7d7b3439f061 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:17.417492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c93f366f-99f3-4377-8e29-7d7b3439f061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:17.487603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1221/2435] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:06:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:17.515059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:17.516664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:17.604658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351f694-c525-4f8e-b028-f63910fb01c7 service glance] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:17.609522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886c3e7-5ff6-4646-9d07-a371b695eb13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:17.610152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c3e7-5ff6-4646-9d07-a371b695eb13 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:17.610477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c3e7-5ff6-4646-9d07-a371b695eb13 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:17.610709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886c3e7-5ff6-4646-9d07-a371b695eb13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:17.619993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886c3e7-5ff6-4646-9d07-a371b695eb13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:17.621641 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886c3e7-5ff6-4646-9d07-a371b695eb13 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:17.631901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1215/2436] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:06:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:06:18.714762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe824-658c-4da2-8709-10f37f5e6296 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:18.715437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe824-658c-4da2-8709-10f37f5e6296 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:18.715437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fe824-658c-4da2-8709-10f37f5e6296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:18.744620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ab3fe824-658c-4da2-8709-10f37f5e6296 None None] MFA Rules not processed for user `dc83779cc4e54be794873d62e3ac9061`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:18.745885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ab3fe824-658c-4da2-8709-10f37f5e6296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:18.821290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:18.854069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1222/2437] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:23.617483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ae47a-bdae-436c-9279-c2600a49664c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:23.617483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ae47a-bdae-436c-9279-c2600a49664c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:23.617483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ae47a-bdae-436c-9279-c2600a49664c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:23.636776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f70ae47a-bdae-436c-9279-c2600a49664c None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:23.638082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f70ae47a-bdae-436c-9279-c2600a49664c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:23.652744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1216/2438] 10.222.0.23 () {58 vars in 967 bytes} [Mon Sep 5 17:06:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:23.673140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fe824-658c-4da2-8709-10f37f5e6296 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:23.686057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fe824-658c-4da2-8709-10f37f5e6296 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:23.691550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:23.692305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:23.692606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:23.692913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:23.702300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:23.704370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:23.716711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1223/2439] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:06:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:06:27.364865 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a13b69-6e40-46ad-bfae-39c1408c4924 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:27.365379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a13b69-6e40-46ad-bfae-39c1408c4924 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:27.365379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a13b69-6e40-46ad-bfae-39c1408c4924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:27.385741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-45a13b69-6e40-46ad-bfae-39c1408c4924 None None] MFA Rules not processed for user `c8b8aa2ae43a4d89ad579a5fdd87d564`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:27.386974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-45a13b69-6e40-46ad-bfae-39c1408c4924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:27.394114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1217/2440] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:06:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:28.002894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:28.065035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33511-b38a-4612-b26d-9720dfe3a80b service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:28.069166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ffa6-9008-4783-b418-dc058240fc25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:28.069834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ffa6-9008-4783-b418-dc058240fc25 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:28.070107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ffa6-9008-4783-b418-dc058240fc25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:28.070385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ffa6-9008-4783-b418-dc058240fc25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:28.072292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2101ffa6-9008-4783-b418-dc058240fc25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:28.148300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ffa6-9008-4783-b418-dc058240fc25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:28.149658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ffa6-9008-4783-b418-dc058240fc25 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:28.161494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1224/2441] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:06:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:06:29.118894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a13b69-6e40-46ad-bfae-39c1408c4924 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:29.128752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a13b69-6e40-46ad-bfae-39c1408c4924 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:29.132407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:29.133246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:29.133609 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:29.134107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:29.135856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:29.151107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:29.152829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:29.166279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1218/2442] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:06:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:06:30.126311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36193c-d411-41db-a732-2dab24797bf1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:30.127539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36193c-d411-41db-a732-2dab24797bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:30.127539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36193c-d411-41db-a732-2dab24797bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:30.147888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fb36193c-d411-41db-a732-2dab24797bf1 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:06:30.149540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fb36193c-d411-41db-a732-2dab24797bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:30.157538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1225/2443] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:06:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:06:30.171115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:30.184028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758ae8a-5f5f-4a8b-babc-45ba46fb10ac service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:30.189419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:30.190027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:30.190365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:30.190812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:30.199416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:30.201040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:30.209647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1219/2444] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:06:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:06:40.142114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=69 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:40.206925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=55 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:06:58.985990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36193c-d411-41db-a732-2dab24797bf1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:06:58.996426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36193c-d411-41db-a732-2dab24797bf1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:06:59.001126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:06:59.001739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:06:59.002017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:06:59.002285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:06:59.004497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:06:59.015288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:06:59.016789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:06:59.025215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1226/2445] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:06:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:07:09.021941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:07:09.072055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:07:09.073781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:07:09.093670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf6e8e-8dcf-49da-9822-71129d4c2bb3 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:07:09.101271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:07:09.101934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:07:09.102669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:07:09.102669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:07:09.104620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:07:09.173960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:07:09.176459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:07:09.187736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1220/2446] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:07:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:07:19.183315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=28 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:07:55.642902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:07:55.665355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11fb6d-89bb-446d-9029-3200ce1356e8 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:07:55.671496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:07:55.672102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:07:55.672413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:07:55.672684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:07:55.674522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:07:55.735247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:07:55.736883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:07:55.746022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1227/2447] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:07:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:08:05.742051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=28 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:08:10.942592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:08:10.952683 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96f6a0-8858-4d5f-af84-7883d4ea8523 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:08:10.956483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:08:10.957093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:08:10.957374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:08:10.957644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:08:10.969376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:08:10.971757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:08:10.988288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1221/2448] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:08:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:08:12.674520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:08:12.677251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:08:12.695596 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2e8a5-9022-4770-9448-01f301250075 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:08:12.702495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558041b4-d361-4805-a040-be93157cd82e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:08:12.703228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558041b4-d361-4805-a040-be93157cd82e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:08:12.703497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558041b4-d361-4805-a040-be93157cd82e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:08:12.703884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558041b4-d361-4805-a040-be93157cd82e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:08:12.705837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-558041b4-d361-4805-a040-be93157cd82e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:08:12.719511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558041b4-d361-4805-a040-be93157cd82e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:08:12.721190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558041b4-d361-4805-a040-be93157cd82e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:08:12.733507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1228/2449] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:08:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:08:14.410284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:08:14.440753 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63430599-e5e6-4f8e-8c8f-9eb2f6ccbb1d service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:08:14.456333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:08:14.457300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:08:14.458196 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:08:14.458656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:08:14.460646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:08:14.476446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:08:14.478385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:08:14.490406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1222/2450] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:08:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:08:22.728881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:08:24.486072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=12 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:08:49.013152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558041b4-d361-4805-a040-be93157cd82e service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:08:49.014601 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-558041b4-d361-4805-a040-be93157cd82e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:08:49.075046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558041b4-d361-4805-a040-be93157cd82e service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:08:49.078782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:08:49.079389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:08:49.079663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:08:49.079928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:08:49.081708 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:08:49.093259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:08:49.094746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:08:49.104820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1229/2451] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:08:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:08:59.100331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=26 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:08:59.635672 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:08:59.645536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba4e62-e2ce-45c9-9cfb-02bb4a0677ce service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:08:59.649695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:08:59.650299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:08:59.650577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:08:59.650850 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:08:59.652883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:08:59.716766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:08:59.718215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:08:59.728842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1223/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:08:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:09:09.723666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=28 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:09:28.655075 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:09:28.672476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8fa96f-4ed6-40d7-b886-01dbd286d021 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:09:28.683151 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:09:28.683943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:09:28.684241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:09:28.684507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:09:28.686452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:09:28.757546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:09:28.759173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:09:28.768402 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1230/2453] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:09:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:09:38.764642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=32 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:10:39.291452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:10:39.293143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:10:39.445679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaf50b-9863-4710-9c3e-0eaccb0b23e8 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:10:39.453746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:10:39.454361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:10:39.454639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:10:39.454898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:10:39.456904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:10:39.468618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:10:39.470194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:10:39.480664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1224/2454] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:10:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:10:49.476472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=63 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:11:17.052640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:11:17.116519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f72da-3fb8-42a3-9949-c1d0b394aa29 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:11:17.122843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:11:17.123459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:11:17.123731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:11:17.124010 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:11:17.132589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:11:17.134153 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:11:17.143399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1231/2455] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:11:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:11:27.139818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=27 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:11:28.557054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:11:28.578427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b2bc1-6d4b-4042-b05e-b7bb8a220557 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:11:28.582780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:11:28.583380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:11:28.583658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:11:28.583925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:11:28.594874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:11:28.596333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:11:28.607715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1225/2456] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:11:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:11:29.726545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:11:29.741779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b134a4-bfcf-42ff-a37e-4e636adcf851 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:11:29.747543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:11:29.748192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:11:29.748483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:11:29.748750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:11:29.761620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:11:29.763055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:11:29.774516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1232/2457] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:11:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:11:38.603067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=10 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:11:39.769383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:12:10.563642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:12:10.565783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:12:10.611762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7c268-7b42-47fc-9f99-06e427ab107d service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:12:10.615369 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:12:10.615977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:12:10.616282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:12:10.616556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:12:10.627110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:12:10.628580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:12:10.657302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1226/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:12:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:12:20.654892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=22 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:12:44.887034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:12:44.896338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea9da0-acf6-45e9-ba1b-1e07c4391c37 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:12:44.900941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:12:44.901547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:12:44.901824 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:12:44.902093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:12:44.910152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:12:44.911597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:12:44.920137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1233/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:12:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:12:54.916509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:12:55.909235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:12:55.929055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04907ba8-f9b4-4cdc-a427-3ac47e6885d8 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:12:55.936591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:12:55.937195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:12:55.937464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:12:55.938018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:12:55.947150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:12:55.948787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:12:55.981298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1227/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:12:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:13:05.978785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=13 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:14:10.933187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:10.948430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68506cc5-ba3c-48a6-b1fb-ffc60464500c service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:10.953457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:10.954061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:10.954340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:10.954607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:10.963507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:10.964989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:10.992783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1234/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:11.035960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:11.046574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261c2f1-fdb5-481e-8280-251a651afbb7 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:11.051789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82962c-dd86-4de4-a31b-0c5ce8a239a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:11.052446 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82962c-dd86-4de4-a31b-0c5ce8a239a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:11.052745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82962c-dd86-4de4-a31b-0c5ce8a239a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:11.053012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82962c-dd86-4de4-a31b-0c5ce8a239a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:11.063561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82962c-dd86-4de4-a31b-0c5ce8a239a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:11.065030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82962c-dd86-4de4-a31b-0c5ce8a239a9 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:11.093215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1228/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:13.855303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:13.873647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d6377-750d-4484-8120-16955263f962 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:13.884965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:13.885921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:13.886403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:13.886826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:13.899354 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:13.901280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:13.927665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1235/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:17.244877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:17.245388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:17.245388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:17.290053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] MFA Rules not processed for user `2b81fb00b1b545308d01f74a55b7e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:14:17.291299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:14:17.299928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1229/2464] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:14:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:14:17.645300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:17.656349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c21b21-2f02-4e00-be51-613c9786aff9 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:17.661342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:17.661943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:17.662209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:17.662482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:17.664515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:14:17.688821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:17.690260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:17.698320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1236/2465] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:18.048197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:18.049734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:14:18.065132 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cf4a3-7d62-44b5-8257-b11a65eb9012 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:18.070708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:18.071331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:18.071613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:18.071888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:18.073712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:14:18.085855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:18.087478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:18.098897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1230/2466] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:25.746882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:25.762060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172dcc4-7198-44c7-ad0b-75ef2d29cecc service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:25.768514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:25.769133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:25.769413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:25.769685 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:25.771416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:14:25.798003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:25.799666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:25.810839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1237/2467] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:28.093706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=30 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:14:29.211210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:29.224754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e349e1-436b-4ce4-89db-5d532be7326a service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:29.231537 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:29.232303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:29.232583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:29.232850 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:29.241513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:29.243233 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:29.271003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1231/2468] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:30.185860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:14:30.197879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e3d74-e179-4731-bc7e-4ecdad3d1462 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:14:30.203967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:14:30.204588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:14:30.204866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:14:30.205139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:14:30.215161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:14:30.216807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:14:30.228647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1238/2469] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:14:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:14:39.268526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=13 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:14:40.223795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=47 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:15:42.277990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:15:42.292452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa785c2-75df-499a-b219-d8a7d3a1a0cf service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:15:42.298199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:15:42.298802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:15:42.299079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:15:42.299350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:15:42.308677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:15:42.310270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:15:42.320689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1232/2470] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:15:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:15:52.316250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:17:05.578139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:17:05.592855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5000378-c5bc-4635-8535-9de21212cd78 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:17:05.598044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:17:05.598659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:17:05.598937 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:17:05.599216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:17:05.601314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:17:05.674721 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:17:05.676087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:17:05.687507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1239/2471] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:17:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:17:05.839463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:17:05.852538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9784ae-fc06-42b1-a91b-b00639bf4c56 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:17:05.858545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:17:05.859162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:17:05.859489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:17:05.859760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:17:05.861665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:17:05.947802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:17:05.957624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:17:05.958306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c92050-91f7-4971-a7d4-245121cfde74 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:17:05.959299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:17:05.962434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:17:05.963146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:17:05.963517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:17:05.963885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:17:05.966722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:17:05.985785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:17:05.986157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1233/2472] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:17:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:17:05.986260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:17:05.998754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1240/2473] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:17:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:17:15.976041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=41 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:17:15.992636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=40 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:17:49.873208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:17:49.874847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:17:49.940606 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc835a-a17b-4b9c-89e0-a2b64890818b service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:17:49.945552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:17:49.946159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:17:49.946444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:17:49.946707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:17:49.948801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:17:49.959402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:17:49.960859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:17:49.969156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1234/2474] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:17:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:17:59.965402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=26 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:18:11.763545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:18:11.773515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d143ff3-f7de-4901-8a48-590893e365d8 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:18:11.777140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:18:11.777754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:18:11.778031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:18:11.778305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:18:11.780959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:18:11.857172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:18:11.858865 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:18:11.868475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1241/2475] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:18:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:18:11.911802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:18:11.925024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969749a-58d2-4100-8914-2a2d80daefb5 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:18:11.929886 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:18:11.930489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:18:11.930769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:18:11.931040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:18:11.933128 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:18:12.007958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:18:12.009419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:18:12.022387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1235/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:18:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:18:21.863873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=33 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:18:22.015104 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=33 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:19:11.066508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:19:11.068945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:19:11.133156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb71bf-25a8-436b-bcc2-dc7b9f597895 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:19:11.139555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:19:11.140200 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:19:11.140486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:19:11.140813 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:19:11.143582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:19:11.230389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:19:11.231759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:19:11.240785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1242/2477] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:19:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:19:11.587771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:19:11.651502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648037bd-f409-49b9-a5d9-49ce85f2229a service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:19:11.655477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:19:11.656083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:19:11.656381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:19:11.656645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:19:11.658576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:19:11.719653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:19:11.721459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:19:11.730318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1236/2478] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:19:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:19:21.237190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=53 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:19:21.726214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=48 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:19:29.207786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:19:29.224542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bdf096-bd83-4ae7-8326-2d3f92788050 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:19:29.230285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:19:29.230891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:19:29.231167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:19:29.231437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:19:29.233591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:19:29.313775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:19:29.315322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:19:29.324677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1243/2479] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:19:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:19:39.320657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=33 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:20:14.929271 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:20:14.931085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:20:14.947940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f78876-c3ec-4c3e-a033-c8fb355b6572 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:20:14.954525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:20:14.955158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:20:14.955437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:20:14.955709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:20:14.957906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:20:14.972681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:20:14.974287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:20:14.988680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1237/2480] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:20:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:20:24.989381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:20:49.215007 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:20:49.302511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5141168e-5f1b-411f-ba22-5ca36fb30d11 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:20:49.307826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:20:49.308490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:20:49.308770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:20:49.309040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:20:49.311030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:20:49.344934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:20:49.346525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:20:49.356473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1244/2481] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:20:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:20:59.352314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=49 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:22:04.548599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:22:04.550762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:22:04.603961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e36b1-1100-40df-ab2f-4dfac2e9907c service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:22:04.608237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:22:04.608807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:22:04.609080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:22:04.609344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:22:04.649879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:22:04.652031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:22:04.683122 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1238/2482] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:22:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:22:05.813262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:22:05.839303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebff4f1-2eb7-4197-9aa6-da4a49730f55 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:22:05.843845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:22:05.844484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:22:05.844761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:22:05.845035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:22:05.856097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:22:05.858444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:22:05.872896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1245/2483] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:22:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:22:13.249120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:22:13.268819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff91e1b-5af4-4e0e-89a7-a11aaed5c568 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:22:13.272884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:22:13.273543 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:22:13.273878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:22:13.274325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:22:13.282638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:22:13.284096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:22:13.293290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1239/2484] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:22:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:22:15.865667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=11 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:22:23.289391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=49 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:22:55.842157 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:22:55.852597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643505e-05d7-48a2-b847-02f2ffb81d02 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:22:55.857533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:22:55.858145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:22:55.858424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:22:55.858694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:22:55.866434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:22:55.867893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:22:55.883628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1246/2485] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:22:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:23:05.879562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:23:12.276215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:23:12.290417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e34af3-b89a-4fa3-9173-c0ee22d226a7 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:23:12.297516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:23:12.298145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:23:12.298420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:23:12.298684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:23:12.314264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:23:12.315876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:23:12.360183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1240/2486] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:23:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:23:22.356770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=13 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:24:13.644933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:24:13.657390 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cf6a0-d85b-4840-a14e-664db79260f1 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:24:13.661494 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:24:13.662097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:13.662375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:13.662653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:13.671778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:24:13.673277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:24:13.701672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1247/2487] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:24:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:24:13.789080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:24:13.804760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa712c-c360-4fbc-9dc2-655b21d559fd service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:24:13.810795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760f2e1-029c-4ad3-9b32-c22ccc4049ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:24:13.811400 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760f2e1-029c-4ad3-9b32-c22ccc4049ba service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:13.811674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760f2e1-029c-4ad3-9b32-c22ccc4049ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:13.811938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760f2e1-029c-4ad3-9b32-c22ccc4049ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:13.821417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760f2e1-029c-4ad3-9b32-c22ccc4049ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:24:13.822848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760f2e1-029c-4ad3-9b32-c22ccc4049ba service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:24:13.853408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1241/2488] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:24:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:24:13.957768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:24:13.972041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96330e54-8447-45de-897f-b6ad46263193 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:24:13.979217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:24:13.979918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:13.980244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:13.980524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:13.990973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:24:13.992487 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:24:14.021691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1248/2489] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:24:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:24:20.215038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:20.215593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:20.215703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:20.256581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] MFA Rules not processed for user `2b81fb00b1b545308d01f74a55b7e77b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:24:20.257804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:24:20.265266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1242/2490] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:24:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:24:20.586454 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:24:20.598625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cbb283-0792-4aae-bdac-45dfa2a238a2 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:24:20.602783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:24:20.603423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:20.603697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:20.603995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:20.606210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:24:20.631764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:24:20.633253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:24:20.641655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1249/2491] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:24:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:24:20.998027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:24:21.000031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:24:21.016316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6cd6cd-6c0d-490a-9277-725dc6eee67a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:24:21.021544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:24:21.022157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:21.022435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:21.022715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:21.026864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:24:21.041522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:24:21.043185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:24:21.060251 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1243/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:24:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:24:30.638199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=37 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:24:30.945040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:24:30.946827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:24:30.976332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8213c7-edef-436f-b94b-3f9882487b96 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:24:30.981803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:24:30.982405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:30.982680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:30.982949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:30.984913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:24:30.997231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:24:30.998823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:24:31.010440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1250/2493] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:24:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:24:31.051321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=30 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:24:34.061639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:24:34.075155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67726599-15fd-4322-bab6-272e51beaf6c service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:24:34.081268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bd5bd-2c09-4d43-a07d-933a28eb4433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:24:34.082147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd5bd-2c09-4d43-a07d-933a28eb4433 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:24:34.082423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd5bd-2c09-4d43-a07d-933a28eb4433 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:24:34.082690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd5bd-2c09-4d43-a07d-933a28eb4433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:24:34.084619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c18bd5bd-2c09-4d43-a07d-933a28eb4433 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:24:34.097468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bd5bd-2c09-4d43-a07d-933a28eb4433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:24:34.099047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bd5bd-2c09-4d43-a07d-933a28eb4433 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:24:34.109451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1244/2494] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:24:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:24:41.006944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=11 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:24:44.105322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:25:56.188105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:25:56.189782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:25:56.206035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b79a11-7ae0-4761-bce2-28cc6790af7e service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:25:56.212893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:25:56.213507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:25:56.213786 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:25:56.214058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:25:56.223779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:25:56.225414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:25:56.235281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1251/2495] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:25:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:26:06.231222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:26:13.469183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e1488-fe6a-4b30-97a5-77bbb045feff None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:13.470053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e1488-fe6a-4b30-97a5-77bbb045feff None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:13.470053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e1488-fe6a-4b30-97a5-77bbb045feff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:13.505086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d05e1488-fe6a-4b30-97a5-77bbb045feff None None] MFA Rules not processed for user `c53ac49fbc054c01a5a1ef41e04bf253`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:26:13.506304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d05e1488-fe6a-4b30-97a5-77bbb045feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:26:13.517515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1245/2496] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:26:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:26:13.881489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:13.892809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a158c-ccb5-4473-acf8-6930940ffbc8 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:13.900074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:13.901656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:13.901941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:13.902215 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:13.904330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:26:13.917418 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:13.919024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:13.932931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1252/2497] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:26:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:26:14.361451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e1488-fe6a-4b30-97a5-77bbb045feff None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:14.376050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e1488-fe6a-4b30-97a5-77bbb045feff None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:14.381326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:14.382477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:14.382990 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:14.383650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:14.407324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:14.409501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:14.446267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1246/2498] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:26:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:26:15.166924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:15.185321 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7de16-8160-4c8c-97e0-5b3a8b0c5a4b service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:15.189939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:15.190137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:15.190624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:15.192042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:15.194661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:26:15.377251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:15.378776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:15.401153 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1253/2499] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:26:18.655995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:18.670612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee1c6-4e2e-4bdd-881e-cfabf201f5e4 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:18.680693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:18.682074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:18.682444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:18.682725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:18.694492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:18.696620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:18.710789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1247/2500] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:26:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:26:20.339870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:20.353602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446fade-19b5-40a3-9c19-304b4c34ae91 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:20.359356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:20.359956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:20.360453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:20.360732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:20.362644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:26:20.380485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:20.382099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:20.392887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1254/2501] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:26:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:26:24.024181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:24.027712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:26:24.042379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b9bf2-8cf8-4059-972f-a811c707f320 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:24.050347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:24.051010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:24.051290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:24.051565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] PATH_INFO: `/v3/users/9b3a59d52ce8432c964f8c94a6e96a4d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:24.052732 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b3a59d52ce8432c964f8c94a6e96a4d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:24.054695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:24.109192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b3a59d52ce8432c964f8c94a6e96a4d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.109827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b3a59d52ce8432c964f8c94a6e96a4d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.122817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b3a59d52ce8432c964f8c94a6e96a4d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.141493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b3a59d52ce8432c964f8c94a6e96a4d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.142335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b3a59d52ce8432c964f8c94a6e96a4d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.156896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1248/2502] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:26:24 2022] DELETE /identity/v3/users/9b3a59d52ce8432c964f8c94a6e96a4d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:26:24.359128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:24.378599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69984c-27ee-4b0c-b71c-a9f32de5b5c4 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:24.447911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:24.448592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:24.448875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:24.449152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] PATH_INFO: `/v3/projects/e6c32d7007a1428a8bde1b73e5883db6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:24.450308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c32d7007a1428a8bde1b73e5883db6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:24.452294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:24.498468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:26:24.505854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c32d7007a1428a8bde1b73e5883db6', 'request_id': 'req-720daec8-e120-4239-9393-b3b0f26ce6f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.513978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c32d7007a1428a8bde1b73e5883db6', 'request_id': 'req-720daec8-e120-4239-9393-b3b0f26ce6f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.519863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c32d7007a1428a8bde1b73e5883db6', 'request_id': 'req-720daec8-e120-4239-9393-b3b0f26ce6f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.534251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] The token cache is being invalidate because project e6c32d7007a1428a8bde1b73e5883db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:26:24.534544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.535306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.536956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1255/2503] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:26:24 2022] DELETE /identity/v3/projects/e6c32d7007a1428a8bde1b73e5883db6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:26:24.541988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:24.543542 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:26:24.557651 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4daf51-eff7-4ba0-beb7-bd3043a92ee0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:24.620098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:24.620737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:24.621011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:24.621307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] PATH_INFO: `/v3/users/ac154588bdd64c58b43c50c7e78e4d14` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:24.634787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac154588bdd64c58b43c50c7e78e4d14)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:24.636722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:24.642402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:26:24.687425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac154588bdd64c58b43c50c7e78e4d14'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.688075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac154588bdd64c58b43c50c7e78e4d14'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.697769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac154588bdd64c58b43c50c7e78e4d14'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.709985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac154588bdd64c58b43c50c7e78e4d14'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.710635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac154588bdd64c58b43c50c7e78e4d14'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:24.723960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1249/2504] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:26:24 2022] DELETE /identity/v3/users/ac154588bdd64c58b43c50c7e78e4d14 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:26:24.897254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:26:24.898837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:26:24.912500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720daec8-e120-4239-9393-b3b0f26ce6f2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:26:24.970693 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:26:24.971334 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:26:24.971607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:26:24.971872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] PATH_INFO: `/v3/projects/c78ebda5da1441abbadbca3820920988` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:26:24.983474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78ebda5da1441abbadbca3820920988)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:26:24.985436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:26:25.035701 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78ebda5da1441abbadbca3820920988', 'request_id': 'req-56d64f5b-96e0-41f8-8a29-9daee31e747c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:25.042637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78ebda5da1441abbadbca3820920988', 'request_id': 'req-56d64f5b-96e0-41f8-8a29-9daee31e747c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:25.048602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78ebda5da1441abbadbca3820920988', 'request_id': 'req-56d64f5b-96e0-41f8-8a29-9daee31e747c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:25.063951 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] The token cache is being invalidate because project c78ebda5da1441abbadbca3820920988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:26:25.064264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:25.064995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:26:25.066592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1256/2505] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:26:24 2022] DELETE /identity/v3/projects/c78ebda5da1441abbadbca3820920988 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:26:34.720521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:26:35.061259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=64,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:27:04.666647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:27:04.668563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:04.729156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e1e2e-6eb8-440e-830b-be5237d156cf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:27:04.779106 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:27:04.779721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:27:04.779999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:27:04.780308 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:27:04.782244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:04.855051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:27:04.856533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:27:04.870366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1250/2506] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:27:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:27:05.799675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:27:05.801415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:05.815342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d64f5b-96e0-41f8-8a29-9daee31e747c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:27:05.820626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:27:05.821243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:27:05.821516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:27:05.821766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:27:05.823798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:05.838562 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:27:05.839985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:27:05.852015 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1257/2507] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:27:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:27:13.057661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:27:13.059363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:13.074139 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5ff4e-b129-4717-891c-e39454c50b2b service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:27:13.080987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:27:13.081604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:27:13.081878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:27:13.082148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:27:13.084049 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:13.143069 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:27:13.168187 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:27:13.169793 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:27:13.189380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1251/2508] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:27:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:27:15.847053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:27:23.186778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=15 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:27:59.847164 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:27:59.849270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:59.866305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0acdf-cd5d-46e3-8227-149cb5c31b26 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:27:59.870904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:27:59.871573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:27:59.871856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:27:59.872110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:27:59.874204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:27:59.884800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:27:59.886267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:27:59.894731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1258/2509] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:27:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:28:09.891005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:28:15.453577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:15.455291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:15.537266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71032945-71a1-41ca-8333-27ae39194acb service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:15.540996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:15.541621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:15.541892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:15.542194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:15.544057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:15.625563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:15.627249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:15.636614 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1252/2510] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:28:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:28:21.709626 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:21.719353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db775f-a5c1-48d2-a19f-16ae080226eb service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:21.723976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:21.724632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:21.724930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:21.725198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:21.727182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:21.794892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:21.796388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:21.806845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1259/2511] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:28:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:28:25.632338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=58 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:28:25.893962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:25.895492 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:25.910252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee1bda-95b9-444a-9077-d8d809b6a39b service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:25.915224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:25.915861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:25.916137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:25.916440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] PATH_INFO: `/v3/users/06e60909a8b04008af659620527e3236` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:25.930875 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e60909a8b04008af659620527e3236)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:25.932772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:25.984337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e60909a8b04008af659620527e3236'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:25.985064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e60909a8b04008af659620527e3236'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:25.995227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e60909a8b04008af659620527e3236'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.008867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e60909a8b04008af659620527e3236'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.009608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cf5ed638-3ed8-454d-894b-8cb79bd7b8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e60909a8b04008af659620527e3236'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.022045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1253/2512] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:28:25 2022] DELETE /identity/v3/users/06e60909a8b04008af659620527e3236 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:28:26.228564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:26.246468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f152f6-b767-4194-a595-522ad23feeab service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:26.321126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:26.321770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.322048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.322322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] PATH_INFO: `/v3/projects/cdbd9f06f3484da6a6fd14977b82f384` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.323442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdbd9f06f3484da6a6fd14977b82f384)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:26.325430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:26.375206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdbd9f06f3484da6a6fd14977b82f384', 'request_id': 'req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.382117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdbd9f06f3484da6a6fd14977b82f384', 'request_id': 'req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.389392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdbd9f06f3484da6a6fd14977b82f384', 'request_id': 'req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.405023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] The token cache is being invalidate because project cdbd9f06f3484da6a6fd14977b82f384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:28:26.405312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.405928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c9dcb262-7f5a-484a-bbe6-7aa2ff0ea473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:28:26.407516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1260/2513] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:28:26 2022] DELETE /identity/v3/projects/cdbd9f06f3484da6a6fd14977b82f384 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:28:26.414041 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc95249-6205-46ad-a25d-79a86d83e3d7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.414324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc95249-6205-46ad-a25d-79a86d83e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.414595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc95249-6205-46ad-a25d-79a86d83e3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.475390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dcc95249-6205-46ad-a25d-79a86d83e3d7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:28:26.476584 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dcc95249-6205-46ad-a25d-79a86d83e3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:26.535281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1254/2514] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:28:26.544915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf82e70-7570-4e10-8f49-b3b146950f3b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.545197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf82e70-7570-4e10-8f49-b3b146950f3b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.545470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf82e70-7570-4e10-8f49-b3b146950f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.566146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-daf82e70-7570-4e10-8f49-b3b146950f3b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:28:26.567442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-daf82e70-7570-4e10-8f49-b3b146950f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:26.575569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1261/2515] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:28:26.579976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc95249-6205-46ad-a25d-79a86d83e3d7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:26.589913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc95249-6205-46ad-a25d-79a86d83e3d7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:26.595278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:26.595876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.596189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.596479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.597130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:28:26.597643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:26.599250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:26.611178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1255/2516] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:28:26.615416 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf82e70-7570-4e10-8f49-b3b146950f3b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:26.624959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf82e70-7570-4e10-8f49-b3b146950f3b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:26.630149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:26.630761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.631021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.631289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.631916 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:26.633889 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:26.649999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:28:26.682665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1262/2517] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:28:26 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:28:26.686957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:26.698671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4a0a5-3dc7-48ae-9191-b12f32a0f938 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:26.713711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:28:26.760613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:26.761141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.761422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.761693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.762340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:26.764313 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:26.789782 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1256/2518] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:28:26 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:28:26.794426 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:26.806885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf98a50-60cf-422a-a6dd-bb26c5cc7be4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:26.811843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:26.812499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.812790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.813049 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.813720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:26.815330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:26.827199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1263/2519] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:28:26.831636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:26.843266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70d6c-07ca-4fce-8f7f-d765eaa17ac0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:26.848250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5988e-cf6f-44e8-9532-259e1d79d311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:26.848864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5988e-cf6f-44e8-9532-259e1d79d311 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.849145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5988e-cf6f-44e8-9532-259e1d79d311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.849418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5988e-cf6f-44e8-9532-259e1d79d311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.850115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5988e-cf6f-44e8-9532-259e1d79d311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:26.851743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5988e-cf6f-44e8-9532-259e1d79d311 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:26.863131 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1257/2520] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:28:26.868026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:26.880406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad661b80-8098-44ed-84aa-90ef50cf503b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:26.883624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:26.884387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.884714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.884997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] PATH_INFO: `/v3/projects/f8dd08c068a5435c8c426d4a9a24b30d/users/bb7767df40d849b8bb870ad090fead83/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.902674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dd08c068a5435c8c426d4a9a24b30d, user_id=bb7767df40d849b8bb870ad090fead83, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:26.905400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:26.919467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1264/2521] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:28:26 2022] PUT /identity/v3/projects/f8dd08c068a5435c8c426d4a9a24b30d/users/bb7767df40d849b8bb870ad090fead83/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:28:26.924698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4de9-9485-4fbc-b551-248e8e17a3cd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:26.925024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4de9-9485-4fbc-b551-248e8e17a3cd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:26.925336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4de9-9485-4fbc-b551-248e8e17a3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:26.944429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3eca4de9-9485-4fbc-b551-248e8e17a3cd None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:28:26.945649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3eca4de9-9485-4fbc-b551-248e8e17a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:27.004410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1258/2522] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:28:27.014588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:27.016730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:27.081634 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd6d2d-34ce-4b2b-99ed-4bb7c1287583 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:27.086516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:27.087131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:27.087406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:27.087667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:27.098124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:27.099581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:27.109639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1265/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:28:31.312301 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f392f01-1e77-4dee-a4c2-fc0adb5b7a85 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:31.312805 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f392f01-1e77-4dee-a4c2-fc0adb5b7a85 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:31.312926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f392f01-1e77-4dee-a4c2-fc0adb5b7a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:31.356204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7f392f01-1e77-4dee-a4c2-fc0adb5b7a85 None None] MFA Rules not processed for user `bb7767df40d849b8bb870ad090fead83`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:28:31.357633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7f392f01-1e77-4dee-a4c2-fc0adb5b7a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:31.442588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1259/2524] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:28:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:28:31.457911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:28:31.460372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:31.547155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04af8-d934-4826-9cc8-72d8db98170a service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:28:31.553272 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:28:31.554020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:31.554413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:31.554761 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:31.564080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:28:31.566462 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:28:31.577339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1266/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:28:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:28:33.028976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:28:33.029552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:28:33.029552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:28:33.049508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] MFA Rules not processed for user `bb7767df40d849b8bb870ad090fead83`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:28:33.051011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:28:33.058408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1260/2526] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:28:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:28:41.572491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:28:43.044625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:29:13.757638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:29:13.777333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c65556-6394-4bb0-b519-26530486cecf service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:29:13.784457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:29:13.785058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:29:13.785327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:29:13.785597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:29:13.787720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:29:13.860841 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:29:13.862574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:29:13.872123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1267/2527] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:29:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:29:23.867675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=33 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:30:15.866770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:30:15.866770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:30:15.941481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:30:15.942997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:30:15.959574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ed392-4111-42b8-abd8-afd9f2dd64fc None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:30:15.963177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:30:15.963818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:30:15.964376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:30:15.964376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:30:15.966499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:30:16.010136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3001a-9d7a-4db7-a115-1eef6f14d4bc service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:30:16.018079 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:30:16.018810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:30:16.019093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:30:16.019365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:30:16.021759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:30:16.036930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:30:16.038370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:30:16.047646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1261/2528] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:30:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:30:16.083508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:30:16.085045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:30:16.094952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1268/2529] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:30:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:30:26.043445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=48 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:30:26.090394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=48 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:30:59.145318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:30:59.146755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:30:59.255111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90015373-84e5-43b0-8e44-ebf23f4e9656 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:30:59.260658 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:30:59.261333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:30:59.261611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:30:59.261876 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:30:59.271919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:30:59.277039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:30:59.279599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:30:59.290813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1262/2530] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:30:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:31:09.286638 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=41 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:32:05.093419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:05.124021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b2331-7ea0-4c80-a8ee-d0e79e47de3c service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:05.128085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:05.128723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:05.129003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:05.129273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:05.131216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:05.187209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:05.188782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:05.211877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1269/2531] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:32:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:32:05.897930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:05.920236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c6a21-eaa6-45ed-adf1-106bd02b3774 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:05.924309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db7772-1e90-48f5-9d08-17921f75879f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:05.924922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db7772-1e90-48f5-9d08-17921f75879f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:05.925195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db7772-1e90-48f5-9d08-17921f75879f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:05.925462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db7772-1e90-48f5-9d08-17921f75879f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:05.927269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e8db7772-1e90-48f5-9d08-17921f75879f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:05.941570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db7772-1e90-48f5-9d08-17921f75879f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:05.943034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db7772-1e90-48f5-9d08-17921f75879f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:05.954982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1263/2532] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:32:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:32:12.987794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:13.012408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d36b9d-7bb7-4388-92f8-7e34e3ec0c5e service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:13.019211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:13.019814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:13.020091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:13.020397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:13.022428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:13.082923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:13.084615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:13.094466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1270/2533] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:32:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:32:15.502867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d236f-a28e-4b0f-b558-32996ffd8632 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:15.503382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d236f-a28e-4b0f-b558-32996ffd8632 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:15.503497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d236f-a28e-4b0f-b558-32996ffd8632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:15.565003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f83d236f-a28e-4b0f-b558-32996ffd8632 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:32:15.566305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f83d236f-a28e-4b0f-b558-32996ffd8632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:15.638137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1264/2534] 10.222.0.23 () {58 vars in 967 bytes} [Mon Sep 5 17:32:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:32:15.679953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:15.681931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:15.768717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:32:15.778590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c4df2-2e9f-4453-933f-d6bb120b1f41 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:15.784318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:15.784978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:15.785263 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:15.785530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:15.796075 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:15.797734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:15.806659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1271/2535] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:32:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:32:16.342923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d236f-a28e-4b0f-b558-32996ffd8632 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:16.353764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d236f-a28e-4b0f-b558-32996ffd8632 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:16.358186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:16.358807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:16.359084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:16.359375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:16.361585 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:16.374887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:16.376372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:16.389050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1265/2536] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:32:18.971021 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:18.973428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:19.012358 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03112018-f385-4b45-8e72-1e7ac13c1140 service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:19.020281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:19.021966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:19.022409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:19.022912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:19.026250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:19.065959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:19.067606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:19.113941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1272/2537] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:32:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:32:20.037643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:20.039904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:20.058445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea397637-9c57-4468-aa5e-a3894e9bc7e8 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:20.070001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:20.070852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:20.071318 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:20.072266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:20.074248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:20.087193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:20.088847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:20.101456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1266/2538] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:32:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:32:23.903527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:23.905147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:23.919026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e88d70-22b8-4469-93f9-65717b9b77cf service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:23.923804 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:23.924480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:23.924758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:23.925031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] PATH_INFO: `/v3/users/c8b8aa2ae43a4d89ad579a5fdd87d564` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:23.926146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b8aa2ae43a4d89ad579a5fdd87d564)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:23.928087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:23.982421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b8aa2ae43a4d89ad579a5fdd87d564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:23.992479 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b8aa2ae43a4d89ad579a5fdd87d564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.005388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b8aa2ae43a4d89ad579a5fdd87d564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.006038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b8aa2ae43a4d89ad579a5fdd87d564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.013190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b8aa2ae43a4d89ad579a5fdd87d564'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.019281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1273/2539] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:32:23 2022] DELETE /identity/v3/users/c8b8aa2ae43a4d89ad579a5fdd87d564 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:32:24.212623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:24.225847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f705ce-c4f7-4cd9-9af5-8299fd53a276 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:24.286089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:24.286731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:24.287007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:24.287279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] PATH_INFO: `/v3/projects/0b2e9de2e0a64e67beb01dc673c653e5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:24.288541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2e9de2e0a64e67beb01dc673c653e5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:24.290494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:24.307455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:32:24.346326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2e9de2e0a64e67beb01dc673c653e5', 'request_id': 'req-efbec0ea-584c-4a0a-8c46-942297909b97'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.352974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2e9de2e0a64e67beb01dc673c653e5', 'request_id': 'req-efbec0ea-584c-4a0a-8c46-942297909b97'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.361616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2e9de2e0a64e67beb01dc673c653e5', 'request_id': 'req-efbec0ea-584c-4a0a-8c46-942297909b97'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.377440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] The token cache is being invalidate because project 0b2e9de2e0a64e67beb01dc673c653e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:32:24.377740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.378406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.379993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1267/2540] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:32:24 2022] DELETE /identity/v3/projects/0b2e9de2e0a64e67beb01dc673c653e5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:32:24.384599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:24.386280 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:24.400648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3cf98-f2fe-481d-887f-5e7902599e4b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:24.458912 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:24.459568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:24.460039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:24.460347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] PATH_INFO: `/v3/users/9e690642cf4947eba950ddead258617f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:24.474006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e690642cf4947eba950ddead258617f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:24.476672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:24.519488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:32:24.530772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e690642cf4947eba950ddead258617f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.540258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e690642cf4947eba950ddead258617f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.552851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e690642cf4947eba950ddead258617f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.553784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e690642cf4947eba950ddead258617f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.561170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e690642cf4947eba950ddead258617f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.567381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1274/2541] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:32:24 2022] DELETE /identity/v3/users/9e690642cf4947eba950ddead258617f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:32:24.751020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:32:24.752562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:32:24.767626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbec0ea-584c-4a0a-8c46-942297909b97 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:32:24.827092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:32:24.828194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:32:24.828479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:32:24.828745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] PATH_INFO: `/v3/projects/f8ff0444abda46338ef8c0c7b1f696cf` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:32:24.839605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ff0444abda46338ef8c0c7b1f696cf)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:32:24.841598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:32:24.890698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ff0444abda46338ef8c0c7b1f696cf', 'request_id': 'req-f7f9db7f-741a-4d52-8de0-f0719c0920bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.896158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ff0444abda46338ef8c0c7b1f696cf', 'request_id': 'req-f7f9db7f-741a-4d52-8de0-f0719c0920bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.903775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ff0444abda46338ef8c0c7b1f696cf', 'request_id': 'req-f7f9db7f-741a-4d52-8de0-f0719c0920bc'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.917890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] The token cache is being invalidate because project f8ff0444abda46338ef8c0c7b1f696cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:32:24.918172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.918821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:32:24.920403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1268/2542] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:32:24 2022] DELETE /identity/v3/projects/f8ff0444abda46338ef8c0c7b1f696cf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:32:34.563980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:32:34.914840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=73,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:33:07.850318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:33:07.852014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:33:07.913588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2765e8-573e-43de-8457-b5d054a3e4dc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:33:07.966136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:33:07.966819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:33:07.967075 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:33:07.967342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:33:07.969322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:33:07.979918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:33:07.981419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:33:07.990273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1275/2543] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:33:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:33:17.986453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=46 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:34:17.339345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:34:17.340963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:34:17.412383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9db7f-741a-4d52-8de0-f0719c0920bc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:34:17.416761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:34:17.417454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:34:17.417734 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:34:17.418015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:34:17.420048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:34:17.496671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:34:17.498300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:34:17.517759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1269/2544] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:34:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:34:17.561782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:34:17.571617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10e504-181a-4177-b67d-3e7b380fb560 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:34:17.575738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:34:17.576365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:34:17.576641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:34:17.576910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:34:17.578773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:34:17.642250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:34:17.643706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:34:17.653367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1276/2545] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:34:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:34:17.697078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:34:17.706160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5534ad-98cf-431e-a012-5033cf419ba5 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:34:17.710066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:34:17.710663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:34:17.710943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:34:17.711211 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:34:17.713154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:34:17.789382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:34:17.790742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:34:17.810241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1270/2546] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:34:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:34:27.649040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=28 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:34:27.807102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:36:04.133014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:36:04.134671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:04.210118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac2c72-eaea-48ec-8259-e42c1f222e37 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:36:04.216811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:36:04.217439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:36:04.217716 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:36:04.217987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:36:04.220026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:04.231900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:36:04.233552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:36:04.244169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1277/2547] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:36:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:36:14.239947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=32 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:36:18.185020 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:36:18.194377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f72a5-ad89-4446-b8c2-37dea8ad4c99 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:36:18.198775 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b24c1-890b-49a8-9d4c-1ce9fd0ecb4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:36:18.199385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b24c1-890b-49a8-9d4c-1ce9fd0ecb4b service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:36:18.199681 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b24c1-890b-49a8-9d4c-1ce9fd0ecb4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:36:18.199928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b24c1-890b-49a8-9d4c-1ce9fd0ecb4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:36:18.201827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-608b24c1-890b-49a8-9d4c-1ce9fd0ecb4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:18.288788 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b24c1-890b-49a8-9d4c-1ce9fd0ecb4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:36:18.290226 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b24c1-890b-49a8-9d4c-1ce9fd0ecb4b service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:36:18.309647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1271/2548] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:36:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:36:21.139660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:36:21.141152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:21.247173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694c40f-9a76-4712-b5eb-782e64b42706 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:36:21.260493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:36:21.261130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:36:21.261621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:36:21.262057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:36:21.264406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:21.281718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:36:21.285578 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:36:21.304893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1278/2549] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:36:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:36:24.837019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:36:24.837570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:36:24.837570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:36:24.857761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] MFA Rules not processed for user `bb7767df40d849b8bb870ad090fead83`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:36:24.858986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:24.866928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1272/2550] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:36:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:36:25.218147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:36:25.220484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:25.253673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9853f01-b7c6-4b8e-b79c-9c9eadd9f967 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:36:25.258354 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:36:25.259166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:36:25.259469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:36:25.259772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:36:25.262707 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:25.274423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:36:25.275866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:36:25.284089 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1279/2551] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:36:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:36:25.642046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:36:25.643598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:25.660105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec2e4b-ba7a-4b10-8869-6a4ea3fd2015 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:36:25.665830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:36:25.666447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:36:25.666718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:36:25.666983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:36:25.668863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:25.681291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:36:25.682878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:36:25.693726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1273/2552] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:36:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:36:34.384264 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:36:34.399105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd268790-049e-48b8-ae1e-40c8e0ba405b service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:36:34.404844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:36:34.405458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:36:34.405738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:36:34.406008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:36:34.407949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:36:34.421496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:36:34.423104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:36:34.434649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1280/2553] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:36:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:36:35.689195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=50 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:36:44.429856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=56 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:08.826772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:08.837866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce410ae6-e3a3-4404-8206-2961c343176e service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:08.842475 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:08.843076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:08.843347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:08.843615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:08.853830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:08.855272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:08.864602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1274/2554] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:18.860504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:20.851150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:20.852625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:20.877039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec809a-8331-4371-9d89-f773db0521dc service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:20.880728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:20.881338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:20.881621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:20.881890 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:20.883773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:20.897314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:20.898762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:20.910023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1281/2555] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:30.905393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=11 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:31.391416 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:31.401248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd572132-bcce-48ac-850f-beb629b938d5 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:31.405111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:31.405704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:31.405980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:31.406248 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:31.407997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:31.421300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:31.422720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:31.433832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1275/2556] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:41.429214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:43.858241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:43.859762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:43.872249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801bfa32-a662-420b-affc-327f016c7513 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:43.875884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:43.876648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:43.876920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:43.877186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:43.878973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:43.964028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:43.965511 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:43.985128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1282/2557] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:44.458579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:44.471032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084d48b-17d7-4efe-83f2-836851b377a3 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:44.476671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:44.477283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:44.477565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:44.477840 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:44.479666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:44.491652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:44.493282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:44.503374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1276/2558] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:44.525585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:44.527110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:44.548289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5a990-12d5-4eff-8d03-5c625ed96e45 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:44.556097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:44.557072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:44.557526 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:44.557975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] PATH_INFO: `/v3/users/dc83779cc4e54be794873d62e3ac9061` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:44.559455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc83779cc4e54be794873d62e3ac9061)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:44.561923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:44.616536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc83779cc4e54be794873d62e3ac9061'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.626385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc83779cc4e54be794873d62e3ac9061'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.640326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc83779cc4e54be794873d62e3ac9061'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.641025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc83779cc4e54be794873d62e3ac9061'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.648170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc83779cc4e54be794873d62e3ac9061'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.654327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1283/2559] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:38:44 2022] DELETE /identity/v3/users/dc83779cc4e54be794873d62e3ac9061 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:44.837500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:44.850776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98b5ad-aeed-4c32-8d38-bdaf203d5841 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:44.911576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:44.912348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:44.912631 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:44.912905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] PATH_INFO: `/v3/projects/375cd031a57847ed99991340bf65d3d3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:44.914036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375cd031a57847ed99991340bf65d3d3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:44.915957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:44.968616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375cd031a57847ed99991340bf65d3d3', 'request_id': 'req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.976178 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375cd031a57847ed99991340bf65d3d3', 'request_id': 'req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.982770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375cd031a57847ed99991340bf65d3d3', 'request_id': 'req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.997957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] The token cache is being invalidate because project 375cd031a57847ed99991340bf65d3d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:38:44.998241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:44.998849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:45.000621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1277/2560] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:38:44 2022] DELETE /identity/v3/projects/375cd031a57847ed99991340bf65d3d3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:52.813741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:52.815445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:52.859468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:52.876930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531483-3a11-4ec0-9f48-78bfc49eeffd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:52.926248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:52.926852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:52.927123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:52.927393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:52.929299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:53.004085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:53.005474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:53.015750 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1284/2561] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:54.994696 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:56.219364 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:56.221082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:56.235380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae42fea-4d7b-4b6e-9e3b-38f9c83749f3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:56.240137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:56.240820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:56.241098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:56.241371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] PATH_INFO: `/v3/users/bb7767df40d849b8bb870ad090fead83` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:56.255220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7767df40d849b8bb870ad090fead83)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:56.257202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:56.305878 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7767df40d849b8bb870ad090fead83'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.306523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7767df40d849b8bb870ad090fead83'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.315833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7767df40d849b8bb870ad090fead83'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.328873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7767df40d849b8bb870ad090fead83'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.329524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7395fd64-c454-4b81-aed7-b90f5961ad54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7767df40d849b8bb870ad090fead83'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.343095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1278/2562] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:38:56 2022] DELETE /identity/v3/users/bb7767df40d849b8bb870ad090fead83 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:56.531646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:56.544933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708fc2d-9386-4a43-8567-da7b08836c08 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:56.604432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:56.605073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:56.605361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:56.605633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] PATH_INFO: `/v3/projects/f8dd08c068a5435c8c426d4a9a24b30d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:56.616737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8dd08c068a5435c8c426d4a9a24b30d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:56.618782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:56.665648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:56.677597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8dd08c068a5435c8c426d4a9a24b30d', 'request_id': 'req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.684861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8dd08c068a5435c8c426d4a9a24b30d', 'request_id': 'req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.691988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8dd08c068a5435c8c426d4a9a24b30d', 'request_id': 'req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.709106 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] The token cache is being invalidate because project f8dd08c068a5435c8c426d4a9a24b30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:38:56.709394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.710133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-525f9b47-23e2-4e7b-b42f-5b499d12ff3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:56.713417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1285/2563] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:38:56 2022] DELETE /identity/v3/projects/f8dd08c068a5435c8c426d4a9a24b30d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:56.718737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b70e-15ed-4693-af13-8155cec909fe None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:56.719080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b70e-15ed-4693-af13-8155cec909fe None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:56.719361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537b70e-15ed-4693-af13-8155cec909fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:56.778800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6537b70e-15ed-4693-af13-8155cec909fe None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:56.780017 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6537b70e-15ed-4693-af13-8155cec909fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:56.841027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1279/2564] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:56.849058 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf888a-0605-4266-88f4-7f81e08e7a19 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:56.849344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf888a-0605-4266-88f4-7f81e08e7a19 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:56.849616 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf888a-0605-4266-88f4-7f81e08e7a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:56.870785 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-baaf888a-0605-4266-88f4-7f81e08e7a19 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:56.872019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-baaf888a-0605-4266-88f4-7f81e08e7a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:56.880392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1286/2565] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:56.884925 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537b70e-15ed-4693-af13-8155cec909fe None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:56.896459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537b70e-15ed-4693-af13-8155cec909fe None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:56.903417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:56.904016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:56.904353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:56.904621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:56.905268 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:38:56.905563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:56.907239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:56.919212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1280/2566] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:38:56.923531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf888a-0605-4266-88f4-7f81e08e7a19 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:56.933127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf888a-0605-4266-88f4-7f81e08e7a19 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:56.938261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:56.938862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:56.939136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:56.939406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:56.940066 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:56.942027 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:56.983587 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1287/2567] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:38:56 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:38:56.987792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:56.998663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc8ad6-b331-4728-bfa4-c62c4424192a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:57.011892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:57.059613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:57.060315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.060685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.061019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.061713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:57.063774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:57.094215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1281/2568] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:38:56 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:38:57.098543 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:57.109064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08671476-2dcd-4e1a-8d5f-f55f3b3bbd9e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:57.113929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:57.114527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.114798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.115065 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.115696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:57.117449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:57.127314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1288/2569] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:38:57.131215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:57.143401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3041540-7d99-480c-aa0b-6f0f5c200ac6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:57.149905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ae3e7-a319-4e1d-bf40-54b15bb3da6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:57.150613 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae3e7-a319-4e1d-bf40-54b15bb3da6f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.150987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae3e7-a319-4e1d-bf40-54b15bb3da6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.151297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae3e7-a319-4e1d-bf40-54b15bb3da6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.152015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ae3e7-a319-4e1d-bf40-54b15bb3da6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:57.153861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ae3e7-a319-4e1d-bf40-54b15bb3da6f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:57.167167 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1282/2570] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:38:57.171547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:57.181903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c123c59-798b-48f2-be7c-eabb4704541f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:57.186666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:57.187294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.187571 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.187845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] PATH_INFO: `/v3/projects/e86ec106cddf4056aee5472be012e740/users/922b991e8be44acb9a091f52846f665d/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.202399 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86ec106cddf4056aee5472be012e740, user_id=922b991e8be44acb9a091f52846f665d, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:57.205108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:57.222169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1289/2571] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:38:57 2022] PUT /identity/v3/projects/e86ec106cddf4056aee5472be012e740/users/922b991e8be44acb9a091f52846f665d/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:57.232817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ecadc-0779-4ef7-bc1a-06d93904d878 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.233338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ecadc-0779-4ef7-bc1a-06d93904d878 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.233464 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ecadc-0779-4ef7-bc1a-06d93904d878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.266493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3a7ecadc-0779-4ef7-bc1a-06d93904d878 None None] MFA Rules not processed for user `922b991e8be44acb9a091f52846f665d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:57.267725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3a7ecadc-0779-4ef7-bc1a-06d93904d878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:57.338730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1283/2572] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:57.356437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:57.359714 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:57.445340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9dc31-dda9-4061-b32a-acdc12fe82a5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:57.452576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daa3f4-c397-4846-8439-da08eca1093b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:57.453198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa3f4-c397-4846-8439-da08eca1093b service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.453468 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa3f4-c397-4846-8439-da08eca1093b service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.453762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa3f4-c397-4846-8439-da08eca1093b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.462908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daa3f4-c397-4846-8439-da08eca1093b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:57.464559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daa3f4-c397-4846-8439-da08eca1093b service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:57.473732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1290/2573] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:57.690920 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ecadc-0779-4ef7-bc1a-06d93904d878 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:57.767591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ecadc-0779-4ef7-bc1a-06d93904d878 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:57.772712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:57.773335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.773621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.773889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] PATH_INFO: `/v3/users/922b991e8be44acb9a091f52846f665d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.775034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922b991e8be44acb9a091f52846f665d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:57.776987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:57.810943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:57.833870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922b991e8be44acb9a091f52846f665d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:57.834601 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922b991e8be44acb9a091f52846f665d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:57.844488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922b991e8be44acb9a091f52846f665d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:57.858450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922b991e8be44acb9a091f52846f665d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:57.859417 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922b991e8be44acb9a091f52846f665d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:57.875356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1284/2574] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:38:57 2022] DELETE /identity/v3/users/922b991e8be44acb9a091f52846f665d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:57.880909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:57.881427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:57.881561 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:57.907618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:57.908874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:57.944159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:57.973621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1291/2575] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:57.983746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:57.985227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:58.050989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3de55d-05ed-4106-8cc8-4d57395e6e96 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.055368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d51378-bcb0-42ef-9f18-bb090835b426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.055982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d51378-bcb0-42ef-9f18-bb090835b426 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.056298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d51378-bcb0-42ef-9f18-bb090835b426 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.056558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d51378-bcb0-42ef-9f18-bb090835b426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.066013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d51378-bcb0-42ef-9f18-bb090835b426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.067413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d51378-bcb0-42ef-9f18-bb090835b426 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.076936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1285/2576] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:58.357401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:58.358983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:58.382550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dab8e1-41b8-4b2b-9080-811a78e9f211 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.388324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.388984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.389289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.389564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] PATH_INFO: `/v3/projects/e86ec106cddf4056aee5472be012e740` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.390723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86ec106cddf4056aee5472be012e740)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.392738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.443834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86ec106cddf4056aee5472be012e740', 'request_id': 'req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:58.450650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86ec106cddf4056aee5472be012e740', 'request_id': 'req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:58.456333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86ec106cddf4056aee5472be012e740', 'request_id': 'req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:58.472409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] The token cache is being invalidate because project e86ec106cddf4056aee5472be012e740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:38:58.472697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:58.473325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2c47a95e-2292-49fd-a7a4-807eba4cb9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:38:58.475119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1292/2577] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:38:58 2022] DELETE /identity/v3/projects/e86ec106cddf4056aee5472be012e740 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:58.480703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85671510-6c81-4b7c-957d-051d4ab78220 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.481228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85671510-6c81-4b7c-957d-051d4ab78220 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.481228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85671510-6c81-4b7c-957d-051d4ab78220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.501491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-85671510-6c81-4b7c-957d-051d4ab78220 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:58.502702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-85671510-6c81-4b7c-957d-051d4ab78220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:58.566582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1286/2578] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:58.576328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb56eae-30ad-4a46-9811-48e219d36bda None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.576624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb56eae-30ad-4a46-9811-48e219d36bda None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.576900 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb56eae-30ad-4a46-9811-48e219d36bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.602245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ddb56eae-30ad-4a46-9811-48e219d36bda None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:58.603534 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ddb56eae-30ad-4a46-9811-48e219d36bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:58.613939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1293/2579] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:58.619766 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85671510-6c81-4b7c-957d-051d4ab78220 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:58.629534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85671510-6c81-4b7c-957d-051d4ab78220 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.634771 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.635468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.635806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.636173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.636885 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:38:58.637391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.638907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.651074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1287/2580] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:38:58.655451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb56eae-30ad-4a46-9811-48e219d36bda None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:58.664863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb56eae-30ad-4a46-9811-48e219d36bda None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.669663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.670294 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.670568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.670839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.671498 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.673447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.704570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1294/2581] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:38:58 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:38:58.708970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:58.720515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eb7f7-2c5f-4a05-96b4-27277fe1c6a8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.760707 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:58.788617 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.789222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.789558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.789827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.790559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.792511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.816834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1288/2582] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:38:58 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:38:58.823251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:58.834906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4aaa-812b-4c02-b78a-067f09ae473c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.840182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.840802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.841082 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.841343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.841978 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.843581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.853628 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1295/2583] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:38:58.857616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:58.867966 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf75315-b4e7-4132-ab90-3862288f5d1a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.873724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf591093-ae01-4d4d-b786-331abbab5985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.874319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf591093-ae01-4d4d-b786-331abbab5985 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.874596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf591093-ae01-4d4d-b786-331abbab5985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.874866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf591093-ae01-4d4d-b786-331abbab5985 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.875504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf591093-ae01-4d4d-b786-331abbab5985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.877153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf591093-ae01-4d4d-b786-331abbab5985 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.887283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1289/2584] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:38:58.891550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:58.901968 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78b7bb-ac8b-40e4-8454-32035c7fe778 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:58.906668 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:58.907318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.907592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.907863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] PATH_INFO: `/v3/projects/2a96c289c2f04a80b6a43302f8dd17cc/users/c0a0359acdf7469eb1ea7664d066528d/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.922731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a96c289c2f04a80b6a43302f8dd17cc, user_id=c0a0359acdf7469eb1ea7664d066528d, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:58.925469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:58.942034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1296/2585] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:38:58 2022] PUT /identity/v3/projects/2a96c289c2f04a80b6a43302f8dd17cc/users/c0a0359acdf7469eb1ea7664d066528d/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:38:58.952687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a097b3-e601-4765-934d-286e71942401 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:58.952970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a097b3-e601-4765-934d-286e71942401 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:58.953241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a097b3-e601-4765-934d-286e71942401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:58.982314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-10a097b3-e601-4765-934d-286e71942401 None None] MFA Rules not processed for user `c0a0359acdf7469eb1ea7664d066528d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:58.983523 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-10a097b3-e601-4765-934d-286e71942401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:59.056214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1290/2586] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:59.061440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.131742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:59.135753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e59afd-8a50-4e78-8617-caa1b56c94d5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.141702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.142768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.143056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.143373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.144063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.147252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.181671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1297/2587] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:38:59 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:38:59.182924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a097b3-e601-4765-934d-286e71942401 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.195360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a097b3-e601-4765-934d-286e71942401 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.251436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.252040 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.252363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.252636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.253294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.255354 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.282716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1291/2588] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:38:59 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:38:59.287422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.299210 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b4290-765a-42d0-9fa4-91d067ba3996 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.304022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.304678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.304959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.305419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.306198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.307831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.319055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1298/2589] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:38:59.323592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.334110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f364-5a63-4a6a-868b-b494a04acc57 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.339435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.340109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.340419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.340690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] PATH_INFO: `/v3/projects/dd1f7ece13d34b7b93b705ab88f21219/users/f5a408fdfd23458cb5106af6f87304e0/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.355282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1f7ece13d34b7b93b705ab88f21219, user_id=f5a408fdfd23458cb5106af6f87304e0, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.358024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.377533 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1292/2590] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:38:59 2022] PUT /identity/v3/projects/dd1f7ece13d34b7b93b705ab88f21219/users/f5a408fdfd23458cb5106af6f87304e0/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:59.380404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.395220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a4ff4-ce32-4b89-a10a-360446dc53ae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.464191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492ae0f-df93-431b-9d13-e11b1fb5d7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.464794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492ae0f-df93-431b-9d13-e11b1fb5d7be admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.465069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492ae0f-df93-431b-9d13-e11b1fb5d7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.465347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492ae0f-df93-431b-9d13-e11b1fb5d7be admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.465972 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492ae0f-df93-431b-9d13-e11b1fb5d7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.467583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492ae0f-df93-431b-9d13-e11b1fb5d7be admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.477589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1299/2591] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:38:59.482173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.493191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b4cfa-78c6-48d1-8683-d2d2c837d1da admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.498674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.499330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.499640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.499903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] PATH_INFO: `/v3/projects/dd1f7ece13d34b7b93b705ab88f21219/users/f5a408fdfd23458cb5106af6f87304e0/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.501915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd1f7ece13d34b7b93b705ab88f21219, user_id=f5a408fdfd23458cb5106af6f87304e0, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.504632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.507722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:59.516660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1293/2592] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:38:59 2022] PUT /identity/v3/projects/dd1f7ece13d34b7b93b705ab88f21219/users/f5a408fdfd23458cb5106af6f87304e0/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:38:59.527018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5720d-563e-4958-b432-20f6c4d64d4c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.527301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5720d-563e-4958-b432-20f6c4d64d4c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.527581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5720d-563e-4958-b432-20f6c4d64d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.559339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-58d5720d-563e-4958-b432-20f6c4d64d4c None None] MFA Rules not processed for user `f5a408fdfd23458cb5106af6f87304e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:38:59.560593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-58d5720d-563e-4958-b432-20f6c4d64d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:59.632592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1300/2593] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:38:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:38:59.644174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.646160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:38:59.706839 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993480a3-62c3-42e6-affb-2012b1171aaa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.710722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.711335 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.711612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.712250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.721535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.722979 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.732684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1294/2594] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:38:59.821546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5720d-563e-4958-b432-20f6c4d64d4c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:38:59.832406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5720d-563e-4958-b432-20f6c4d64d4c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:38:59.837361 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505ed5d-52ae-407f-a7e1-0be0373ac569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:38:59.838241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505ed5d-52ae-407f-a7e1-0be0373ac569 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:38:59.838516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505ed5d-52ae-407f-a7e1-0be0373ac569 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:38:59.838783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505ed5d-52ae-407f-a7e1-0be0373ac569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:38:59.893341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:38:59.904181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505ed5d-52ae-407f-a7e1-0be0373ac569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:38:59.905564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505ed5d-52ae-407f-a7e1-0be0373ac569 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:38:59.915131 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1301/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:38:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:09.728286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=28 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:09.910542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:11.060174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:11.126065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d75af4-bde9-4c54-be6b-d501651b03aa service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:11.131263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:11.131890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:11.132183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:11.132462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] PATH_INFO: `/v3/users/c0a0359acdf7469eb1ea7664d066528d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:11.133596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a0359acdf7469eb1ea7664d066528d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:11.135583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:11.187604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a0359acdf7469eb1ea7664d066528d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.188280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a0359acdf7469eb1ea7664d066528d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.197160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a0359acdf7469eb1ea7664d066528d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.209340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a0359acdf7469eb1ea7664d066528d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.209985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a0359acdf7469eb1ea7664d066528d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.222559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1295/2596] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:11 2022] DELETE /identity/v3/users/c0a0359acdf7469eb1ea7664d066528d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:11.227228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:11.227768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:11.227768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:11.246996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:11.248183 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:11.315048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1302/2597] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:11.325860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:11.327671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:11.389059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e2093-5e3a-48ba-bef6-b10aafd85d6b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:11.393157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:11.393773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:11.394046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:11.394310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:11.403620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:11.405074 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:11.414809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1296/2598] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:11.698113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:11.699610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:11.713481 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd1875-9fca-4d10-8214-7beacb61fb31 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:11.718291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:11.718936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:11.719217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:11.719509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] PATH_INFO: `/v3/projects/2a96c289c2f04a80b6a43302f8dd17cc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:11.720918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a96c289c2f04a80b6a43302f8dd17cc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:11.722896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:11.773751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a96c289c2f04a80b6a43302f8dd17cc', 'request_id': 'req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.780867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a96c289c2f04a80b6a43302f8dd17cc', 'request_id': 'req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.787284 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a96c289c2f04a80b6a43302f8dd17cc', 'request_id': 'req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.801797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] The token cache is being invalidate because project 2a96c289c2f04a80b6a43302f8dd17cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:11.802094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.802734 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.804343 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1303/2599] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:11 2022] DELETE /identity/v3/projects/2a96c289c2f04a80b6a43302f8dd17cc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:11.808547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:11.819355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de651342-7ec1-4b1f-b3f6-b5706c4a0045 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:11.841260 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:11.881089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:11.881737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:11.882016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:11.882289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] PATH_INFO: `/v3/users/f5a408fdfd23458cb5106af6f87304e0` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:11.883426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a408fdfd23458cb5106af6f87304e0)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:11.885445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:11.939801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a408fdfd23458cb5106af6f87304e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.940474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a408fdfd23458cb5106af6f87304e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.950356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a408fdfd23458cb5106af6f87304e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.962991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a408fdfd23458cb5106af6f87304e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.963717 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0c06cbd4-2354-45bd-bd11-cdb6af778ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a408fdfd23458cb5106af6f87304e0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:11.976089 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1297/2600] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:11 2022] DELETE /identity/v3/users/f5a408fdfd23458cb5106af6f87304e0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:12.249928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:12.251512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:12.265002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524af37-a94d-4f19-a39c-4bc5c2c6e81a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:12.323108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:12.323759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.324039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.324353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] PATH_INFO: `/v3/projects/dd1f7ece13d34b7b93b705ab88f21219` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.325480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd1f7ece13d34b7b93b705ab88f21219)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:12.327447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:12.360768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:12.379100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd1f7ece13d34b7b93b705ab88f21219', 'request_id': 'req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:12.386434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd1f7ece13d34b7b93b705ab88f21219', 'request_id': 'req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:12.392762 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd1f7ece13d34b7b93b705ab88f21219', 'request_id': 'req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:12.407814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] The token cache is being invalidate because project dd1f7ece13d34b7b93b705ab88f21219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:12.408113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:12.408764 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a8f87ae3-8f96-4395-a0b3-7a30cb70d7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:12.410276 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1304/2601] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:12 2022] DELETE /identity/v3/projects/dd1f7ece13d34b7b93b705ab88f21219 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:12.415304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601159-08cb-4688-94fd-6d01cf7246f2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.415832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601159-08cb-4688-94fd-6d01cf7246f2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.415993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601159-08cb-4688-94fd-6d01cf7246f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.436166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ac601159-08cb-4688-94fd-6d01cf7246f2 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:12.437366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ac601159-08cb-4688-94fd-6d01cf7246f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:12.495374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1298/2602] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:39:12.503755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2aed-3809-4433-94e1-e9466473fe5c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.504039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2aed-3809-4433-94e1-e9466473fe5c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.504350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd2aed-3809-4433-94e1-e9466473fe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.525557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f5bd2aed-3809-4433-94e1-e9466473fe5c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:12.526768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f5bd2aed-3809-4433-94e1-e9466473fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:12.535635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1305/2603] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:12.540048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601159-08cb-4688-94fd-6d01cf7246f2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:12.550361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601159-08cb-4688-94fd-6d01cf7246f2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:12.555500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:12.556096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.556404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.556672 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.557315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:39:12.557616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:12.559224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:12.571505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1299/2604] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:39:12.575993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2aed-3809-4433-94e1-e9466473fe5c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:12.585946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd2aed-3809-4433-94e1-e9466473fe5c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:12.592554 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:12.593162 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.593442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.593717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.594351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:12.596433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:12.627870 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1306/2605] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:12 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:12.632004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:12.642254 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bc888-8dd2-47e9-87cf-68c945c27b4a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:12.672204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:12.706161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:12.706758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.707037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.707340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.707972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:12.710107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:12.732105 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1300/2606] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:12 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:12.736489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:12.748406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48f0bf-29aa-45f1-a2bf-76a4ff026599 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:12.753991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:12.754606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.754892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.755167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.755799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:12.757497 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:12.768363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1307/2607] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:12.772463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:12.782736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6aa38-211b-4419-99a5-eec3739413d9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:12.788344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea7988-712a-4958-bebc-5cd3cb3bbb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:12.788934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7988-712a-4958-bebc-5cd3cb3bbb39 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.789206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7988-712a-4958-bebc-5cd3cb3bbb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.789472 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7988-712a-4958-bebc-5cd3cb3bbb39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.790142 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea7988-712a-4958-bebc-5cd3cb3bbb39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:12.791890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea7988-712a-4958-bebc-5cd3cb3bbb39 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:12.802545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1301/2608] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:12.807002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:12.817421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e9347-2769-4e59-8a74-3ec379cb7e14 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:12.822444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:12.823125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.823405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.823678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] PATH_INFO: `/v3/projects/e950afcca6044f72b94daa9066158a21/users/32467f625c584896b3a083b92128e95b/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.839086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e950afcca6044f72b94daa9066158a21, user_id=32467f625c584896b3a083b92128e95b, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:12.841814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:12.857278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1308/2609] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:12 2022] PUT /identity/v3/projects/e950afcca6044f72b94daa9066158a21/users/32467f625c584896b3a083b92128e95b/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:12.867961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af6c25-7d33-433c-9284-31db1f6c579f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:12.868294 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af6c25-7d33-433c-9284-31db1f6c579f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:12.868596 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af6c25-7d33-433c-9284-31db1f6c579f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:12.897848 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d7af6c25-7d33-433c-9284-31db1f6c579f None None] MFA Rules not processed for user `32467f625c584896b3a083b92128e95b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:12.899067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d7af6c25-7d33-433c-9284-31db1f6c579f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:12.964564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1302/2610] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:39:12.969802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:13.034742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4d9a1-9944-4f69-9ebc-f06d9f462da0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:13.040112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:13.041025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.041302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.041566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.042191 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:13.044364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:13.073756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1309/2611] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:12 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:13.077931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af6c25-7d33-433c-9284-31db1f6c579f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:13.090133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af6c25-7d33-433c-9284-31db1f6c579f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:13.158949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:13.159706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.159987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.160289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.160919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:13.162863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:13.186743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1303/2612] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:13 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:39:13.192540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:13.205627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b468aa5-7fad-4e3c-a71b-863506e99d69 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:13.213717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:13.214482 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.214905 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.215167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.215936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:13.218129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:13.230832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1310/2613] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:13.235168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:13.245586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ec8af-944d-4cbd-9f9a-bfc9570ece7f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:13.250688 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:13.251341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.251619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.251887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] PATH_INFO: `/v3/projects/831f47986a6c4a2a944f7bd0571feb1a/users/459ee00becad428aa3514374fb28f9b1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.267261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=831f47986a6c4a2a944f7bd0571feb1a, user_id=459ee00becad428aa3514374fb28f9b1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:13.270157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:13.289082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1304/2614] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:13 2022] PUT /identity/v3/projects/831f47986a6c4a2a944f7bd0571feb1a/users/459ee00becad428aa3514374fb28f9b1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:13.293097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:13.303953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c977120-63c2-4d05-a9da-8eec92e6088e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:13.326060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:13.372406 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1997079-2448-4dbe-88f4-aa8737fcd518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:13.373000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1997079-2448-4dbe-88f4-aa8737fcd518 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.373291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1997079-2448-4dbe-88f4-aa8737fcd518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.373558 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1997079-2448-4dbe-88f4-aa8737fcd518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.374220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1997079-2448-4dbe-88f4-aa8737fcd518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:13.375856 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1997079-2448-4dbe-88f4-aa8737fcd518 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:13.386620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1311/2615] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:13.391519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:13.400016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:13.407165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3dd3f-e496-43f0-a4e9-82746765f452 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:13.416456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:13.417095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.417470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.417838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] PATH_INFO: `/v3/projects/831f47986a6c4a2a944f7bd0571feb1a/users/459ee00becad428aa3514374fb28f9b1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.421256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=831f47986a6c4a2a944f7bd0571feb1a, user_id=459ee00becad428aa3514374fb28f9b1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:13.425485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:13.436348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1305/2616] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:13 2022] PUT /identity/v3/projects/831f47986a6c4a2a944f7bd0571feb1a/users/459ee00becad428aa3514374fb28f9b1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:39:13.505047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f99556-f66a-4643-90d2-f9e185e91ca0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.505421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f99556-f66a-4643-90d2-f9e185e91ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.505700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f99556-f66a-4643-90d2-f9e185e91ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.537170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e6f99556-f66a-4643-90d2-f9e185e91ca0 None None] MFA Rules not processed for user `459ee00becad428aa3514374fb28f9b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:13.538472 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e6f99556-f66a-4643-90d2-f9e185e91ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:13.611246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1312/2617] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:13.624951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:13.627115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:13.687961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731af4c-7f18-4b3c-bd74-e511164e8ad1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:13.692559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c48772-98df-4f59-9f3f-2a9885d198e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:13.693176 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c48772-98df-4f59-9f3f-2a9885d198e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:13.693451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c48772-98df-4f59-9f3f-2a9885d198e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:13.693721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c48772-98df-4f59-9f3f-2a9885d198e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:13.703504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c48772-98df-4f59-9f3f-2a9885d198e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:13.704956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c48772-98df-4f59-9f3f-2a9885d198e9 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:13.715048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1306/2618] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:14.139699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f99556-f66a-4643-90d2-f9e185e91ca0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:14.205818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f99556-f66a-4643-90d2-f9e185e91ca0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:14.210339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:14.210971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:14.211249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:14.211524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] PATH_INFO: `/v3/users/32467f625c584896b3a083b92128e95b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:14.213022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32467f625c584896b3a083b92128e95b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:14.215018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:14.264360 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32467f625c584896b3a083b92128e95b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.266028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:14.273621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32467f625c584896b3a083b92128e95b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.286293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32467f625c584896b3a083b92128e95b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.286931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32467f625c584896b3a083b92128e95b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.293776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32467f625c584896b3a083b92128e95b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.299642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1313/2619] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:14 2022] DELETE /identity/v3/users/32467f625c584896b3a083b92128e95b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:14.304328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:14.304612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:14.304903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:14.323939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:14.325183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:14.394048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1307/2620] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:39:14.404508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:14.406055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:14.468185 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923be1d-3a20-48c1-b3b6-65fe4963c547 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:14.472574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:14.473199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:14.473485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:14.473757 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:14.483566 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:14.485048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:14.495572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1314/2621] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:14.768095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:14.769610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:14.782289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b671cf-3199-4a6c-be80-8299cad91bc6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:14.787491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:14.788143 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:14.788450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:14.788721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] PATH_INFO: `/v3/projects/e950afcca6044f72b94daa9066158a21` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:14.789852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e950afcca6044f72b94daa9066158a21)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:14.791780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:14.845845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e950afcca6044f72b94daa9066158a21', 'request_id': 'req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.851958 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e950afcca6044f72b94daa9066158a21', 'request_id': 'req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.858296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e950afcca6044f72b94daa9066158a21', 'request_id': 'req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.873463 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] The token cache is being invalidate because project e950afcca6044f72b94daa9066158a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:14.873757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.874395 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:14.875917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1308/2622] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:14 2022] DELETE /identity/v3/projects/e950afcca6044f72b94daa9066158a21 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:14.881338 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:14.891459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c704c42-23bc-47bd-bd71-8bb82faad4c3 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:14.962409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:14.963261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:14.963573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:14.964004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] PATH_INFO: `/v3/users/459ee00becad428aa3514374fb28f9b1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:14.966009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=459ee00becad428aa3514374fb28f9b1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:14.968964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:15.042407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '459ee00becad428aa3514374fb28f9b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.052597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '459ee00becad428aa3514374fb28f9b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.063500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:15.065195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '459ee00becad428aa3514374fb28f9b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.066244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '459ee00becad428aa3514374fb28f9b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.075460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da075cc3-5fe0-4e15-925f-1c592947b323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '459ee00becad428aa3514374fb28f9b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.081477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1315/2623] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:14 2022] DELETE /identity/v3/users/459ee00becad428aa3514374fb28f9b1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:39:15.369722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:15.372278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:15.391983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed80cd1-209d-4b38-9dd0-0dee8acb1e36 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:15.406502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:15.455479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:15.456111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.456428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.456695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] PATH_INFO: `/v3/projects/831f47986a6c4a2a944f7bd0571feb1a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.457797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=831f47986a6c4a2a944f7bd0571feb1a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:15.459700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:15.517738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '831f47986a6c4a2a944f7bd0571feb1a', 'request_id': 'req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.523754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '831f47986a6c4a2a944f7bd0571feb1a', 'request_id': 'req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.529948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '831f47986a6c4a2a944f7bd0571feb1a', 'request_id': 'req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.543679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] The token cache is being invalidate because project 831f47986a6c4a2a944f7bd0571feb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:15.543964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.544595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf86c1b-65c5-4a50-b79e-acb2013a5a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:15.546094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1309/2624] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:15 2022] DELETE /identity/v3/projects/831f47986a6c4a2a944f7bd0571feb1a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:15.551248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec0959-aebc-47bc-bc94-0d9da1b00e87 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.551632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec0959-aebc-47bc-bc94-0d9da1b00e87 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.551878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec0959-aebc-47bc-bc94-0d9da1b00e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.574229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-84ec0959-aebc-47bc-bc94-0d9da1b00e87 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:15.575529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-84ec0959-aebc-47bc-bc94-0d9da1b00e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:15.643794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1316/2625] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:15.652107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc0a1d-89e0-4e3b-bcde-fe26902cedf6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.652431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc0a1d-89e0-4e3b-bcde-fe26902cedf6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.652709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc0a1d-89e0-4e3b-bcde-fe26902cedf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.675156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6dbc0a1d-89e0-4e3b-bcde-fe26902cedf6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:15.676448 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6dbc0a1d-89e0-4e3b-bcde-fe26902cedf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:15.686761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1310/2626] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:15.691583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec0959-aebc-47bc-bc94-0d9da1b00e87 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:15.702148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec0959-aebc-47bc-bc94-0d9da1b00e87 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:15.708064 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:15.708664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.708938 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.709207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.709851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:39:15.710152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:15.711763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:15.723851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1317/2627] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 05 17:39:15.728852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc0a1d-89e0-4e3b-bcde-fe26902cedf6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:15.744307 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc0a1d-89e0-4e3b-bcde-fe26902cedf6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:15.753312 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:15.755743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.756027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.756344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.756978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:15.758890 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:15.791321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1311/2628] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:15 2022] POST /identity/v3/projects => generated 363 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:15.795572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:15.806564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83030748-b728-4b96-b267-87c4cd45f2f4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:15.868133 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:15.868780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.869053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.869318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.869940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:15.871867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:15.896647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1318/2629] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:15 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:15.900961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:15.911476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68340d-928b-48b7-a28d-304ea88783f4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:15.916424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:15.917026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.917302 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.917576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.918222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:15.919834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:15.929984 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1312/2630] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:15.933906 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:15.944635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29e258-e381-4335-a5c6-f8e9cbabbe20 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:15.950204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa114b8b-6d07-4778-a83f-d1e2d1bf2183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:15.950801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114b8b-6d07-4778-a83f-d1e2d1bf2183 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.951073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114b8b-6d07-4778-a83f-d1e2d1bf2183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.951348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa114b8b-6d07-4778-a83f-d1e2d1bf2183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:15.951974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa114b8b-6d07-4778-a83f-d1e2d1bf2183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:15.953635 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa114b8b-6d07-4778-a83f-d1e2d1bf2183 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:15.965600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1319/2631] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:15.971282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:15.983358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05b111-f012-4d2c-ace9-8c7c2ae7463b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:15.988800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:15.989456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:15.989740 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:15.990008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] PATH_INFO: `/v3/projects/1262e713dc534369ba8a26cdfba95391/users/7355712de9c3415292c362ca1b6eadf2/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.007340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1262e713dc534369ba8a26cdfba95391, user_id=7355712de9c3415292c362ca1b6eadf2, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.010180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.027697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1313/2632] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:15 2022] PUT /identity/v3/projects/1262e713dc534369ba8a26cdfba95391/users/7355712de9c3415292c362ca1b6eadf2/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:16.038754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae36c2-eff2-4451-8af1-e8244f600a29 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.039022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae36c2-eff2-4451-8af1-e8244f600a29 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.039297 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae36c2-eff2-4451-8af1-e8244f600a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.069387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ffae36c2-eff2-4451-8af1-e8244f600a29 None None] MFA Rules not processed for user `7355712de9c3415292c362ca1b6eadf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:16.070737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ffae36c2-eff2-4451-8af1-e8244f600a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:16.116259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:16.141257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1320/2633] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:16.146477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.194997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:16.214181 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20f0a3-c1c0-42bf-8dc1-ac68ecc42270 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.219522 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.220123 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.220450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.220706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.221381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.223965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.252453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1314/2634] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:16 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:16.260629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae36c2-eff2-4451-8af1-e8244f600a29 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.271691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae36c2-eff2-4451-8af1-e8244f600a29 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.328446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.329063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.329340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.329614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.330254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.332450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.358440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1321/2635] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:16 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:16.365731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.377415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b135ce0-f545-4a69-9ca5-f1f8d1642a6b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.383212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.383923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.384249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.384518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.385190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.386870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.398483 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1315/2636] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:16.403452 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.416211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad67da-d956-4a37-b7bf-e5317ef6ff59 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.421337 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.422123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.422401 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.422671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] PATH_INFO: `/v3/projects/efc26f4f061a40e38529897447c7a0a7/users/b8fb6e1cac45449a88c89dcaf481dee1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.438425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc26f4f061a40e38529897447c7a0a7, user_id=b8fb6e1cac45449a88c89dcaf481dee1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.441319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.459348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1322/2637] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:16 2022] PUT /identity/v3/projects/efc26f4f061a40e38529897447c7a0a7/users/b8fb6e1cac45449a88c89dcaf481dee1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:16.465286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.475957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66591d7-50ab-44d8-bdfc-2c156b8af5ff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.542144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f5f34-617c-4f88-89b0-aa58015b7ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.542759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f5f34-617c-4f88-89b0-aa58015b7ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.543035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f5f34-617c-4f88-89b0-aa58015b7ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.543336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f5f34-617c-4f88-89b0-aa58015b7ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.543967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f5f34-617c-4f88-89b0-aa58015b7ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.545661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f5f34-617c-4f88-89b0-aa58015b7ed6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.556048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1316/2638] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:39:16.560823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.572292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeac692-7e2f-455c-a945-fe28700079c8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.578271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.578942 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.579223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.579500 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] PATH_INFO: `/v3/projects/efc26f4f061a40e38529897447c7a0a7/users/b8fb6e1cac45449a88c89dcaf481dee1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.581709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc26f4f061a40e38529897447c7a0a7, user_id=b8fb6e1cac45449a88c89dcaf481dee1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.584649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.596530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1323/2639] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:16 2022] PUT /identity/v3/projects/efc26f4f061a40e38529897447c7a0a7/users/b8fb6e1cac45449a88c89dcaf481dee1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:16.608458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81c88-5d42-478f-9270-78fd88f928ba None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.608674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81c88-5d42-478f-9270-78fd88f928ba None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.608947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf81c88-5d42-478f-9270-78fd88f928ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.639110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-caf81c88-5d42-478f-9270-78fd88f928ba None None] MFA Rules not processed for user `b8fb6e1cac45449a88c89dcaf481dee1`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:16.640330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-caf81c88-5d42-478f-9270-78fd88f928ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:16.713162 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1317/2640] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:16.727126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.729259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:16.791770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85749339-09ce-465f-a2ce-0e62dc031219 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.796282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.796896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.797173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.797429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.807504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.808981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.819014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1324/2641] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:16.902116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf81c88-5d42-478f-9270-78fd88f928ba None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:16.912497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf81c88-5d42-478f-9270-78fd88f928ba None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:16.916109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:16.916726 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:16.917000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:16.917272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:16.951624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:16.981258 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:16.982634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:16.992685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1318/2642] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:18.347507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:18.357292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6ae9a-0718-4bee-abbb-49b6f62097d3 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:18.361678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab07590-3905-4c37-9bda-aac952b58cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:18.362432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07590-3905-4c37-9bda-aac952b58cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:18.362770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07590-3905-4c37-9bda-aac952b58cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:18.363043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab07590-3905-4c37-9bda-aac952b58cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:18.365833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4ab07590-3905-4c37-9bda-aac952b58cf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:18.413546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:18.450613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07590-3905-4c37-9bda-aac952b58cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:18.452055 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab07590-3905-4c37-9bda-aac952b58cf1 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:18.462955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1325/2643] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:18 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:19.799355 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:19.863030 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6860629-71b7-4840-add1-25b70f1253a8 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:19.868381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:19.869007 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:19.869284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:19.869541 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] PATH_INFO: `/v3/users/7355712de9c3415292c362ca1b6eadf2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:19.870687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7355712de9c3415292c362ca1b6eadf2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:19.872653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:19.924837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7355712de9c3415292c362ca1b6eadf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:19.925636 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7355712de9c3415292c362ca1b6eadf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:19.935409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7355712de9c3415292c362ca1b6eadf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:19.948914 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7355712de9c3415292c362ca1b6eadf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:19.949560 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7355712de9c3415292c362ca1b6eadf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:19.963259 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1319/2644] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:19 2022] DELETE /identity/v3/users/7355712de9c3415292c362ca1b6eadf2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:19.968130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:19.968638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:19.968720 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:19.992425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:19.993672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:20.068680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1326/2645] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:20.078670 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:20.080184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:20.144931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea56107-d26a-4307-9714-d3777154fa3d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:20.149452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:20.150067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:20.150340 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:20.150605 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:20.161071 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:20.162497 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:20.166372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:20.173480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1320/2646] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:20.455255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:20.456837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:20.470117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0a8a6-3673-4d3a-984a-b6fcf5838af4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:20.475476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:20.476135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:20.476447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:20.476726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] PATH_INFO: `/v3/projects/1262e713dc534369ba8a26cdfba95391` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:20.478211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1262e713dc534369ba8a26cdfba95391)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:20.479970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:20.532525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1262e713dc534369ba8a26cdfba95391', 'request_id': 'req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.538748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1262e713dc534369ba8a26cdfba95391', 'request_id': 'req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.544630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1262e713dc534369ba8a26cdfba95391', 'request_id': 'req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.559673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] The token cache is being invalidate because project 1262e713dc534369ba8a26cdfba95391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:20.559961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.560620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.562136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1327/2647] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:20 2022] DELETE /identity/v3/projects/1262e713dc534369ba8a26cdfba95391 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:20.566272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:20.576236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76df06b-11cc-42cb-8eec-5d76f3b8962c service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:20.636574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:20.637232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:20.637511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:20.637780 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] PATH_INFO: `/v3/users/b8fb6e1cac45449a88c89dcaf481dee1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:20.639023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8fb6e1cac45449a88c89dcaf481dee1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:20.640994 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:20.691837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8fb6e1cac45449a88c89dcaf481dee1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.692579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8fb6e1cac45449a88c89dcaf481dee1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.703199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8fb6e1cac45449a88c89dcaf481dee1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.715654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8fb6e1cac45449a88c89dcaf481dee1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.716425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17578a3e-18fb-4175-b3ba-d43798b61f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8fb6e1cac45449a88c89dcaf481dee1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:20.728534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1321/2648] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:20 2022] DELETE /identity/v3/users/b8fb6e1cac45449a88c89dcaf481dee1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:39:21.006211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.007882 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:21.021369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea472b-9f2b-4178-ab8c-63fb5b2de78b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.067170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:21.081351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.082005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.082281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.082584 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] PATH_INFO: `/v3/projects/efc26f4f061a40e38529897447c7a0a7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.083939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc26f4f061a40e38529897447c7a0a7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.087645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.142140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc26f4f061a40e38529897447c7a0a7', 'request_id': 'req-cb9e2e5f-af80-4779-a50c-b438218de9b8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:21.148848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc26f4f061a40e38529897447c7a0a7', 'request_id': 'req-cb9e2e5f-af80-4779-a50c-b438218de9b8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:21.155447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc26f4f061a40e38529897447c7a0a7', 'request_id': 'req-cb9e2e5f-af80-4779-a50c-b438218de9b8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:21.170167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] The token cache is being invalidate because project efc26f4f061a40e38529897447c7a0a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:21.170462 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:21.171148 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb9e2e5f-af80-4779-a50c-b438218de9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:21.172800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1328/2649] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:21 2022] DELETE /identity/v3/projects/efc26f4f061a40e38529897447c7a0a7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:21.178443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242f88a-1ce1-415f-b800-49d2df2623ce None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.178930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242f88a-1ce1-415f-b800-49d2df2623ce None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.179011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d242f88a-1ce1-415f-b800-49d2df2623ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.200009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d242f88a-1ce1-415f-b800-49d2df2623ce None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:21.201341 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d242f88a-1ce1-415f-b800-49d2df2623ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:21.269180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1322/2650] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:21.277958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473f820-d601-4ec9-853c-cb0697ae35f0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.278270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473f820-d601-4ec9-853c-cb0697ae35f0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.278556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473f820-d601-4ec9-853c-cb0697ae35f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.300822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5473f820-d601-4ec9-853c-cb0697ae35f0 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:21.302121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5473f820-d601-4ec9-853c-cb0697ae35f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:21.311398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1329/2651] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:21.315859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242f88a-1ce1-415f-b800-49d2df2623ce None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.326470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d242f88a-1ce1-415f-b800-49d2df2623ce None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.332166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.332887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.333149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.333444 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.334091 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:39:21.334403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.336009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.350028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1323/2652] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:39:21.354408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473f820-d601-4ec9-853c-cb0697ae35f0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.363875 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473f820-d601-4ec9-853c-cb0697ae35f0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.368799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.369410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.369684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.369958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.370598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.372569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.402945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1330/2653] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:21 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:21.408278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.414867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:21.418727 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d18d8-7bcd-4aab-b517-a46d814ad37f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.481079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.481704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.481973 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.482238 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.482879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.484874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.508992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1324/2654] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:21 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:39:21.513158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.523982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9abbf-db22-4249-9173-54a5b2f81962 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.529435 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.530043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.530326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.530602 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.531240 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.532901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.543484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1331/2655] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:21.547756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.558372 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c9f49-03b3-4903-b7e7-e1c5bf0058db admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.563551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e036328-cffa-4ae1-b5b9-6fbee66ac0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.564165 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036328-cffa-4ae1-b5b9-6fbee66ac0eb admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.564458 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036328-cffa-4ae1-b5b9-6fbee66ac0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.564731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036328-cffa-4ae1-b5b9-6fbee66ac0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.565358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e036328-cffa-4ae1-b5b9-6fbee66ac0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.566965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e036328-cffa-4ae1-b5b9-6fbee66ac0eb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.577819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1325/2656] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:21.581945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.593150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416d422-676d-445d-81d3-01d965db5190 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.598104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.598985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.599270 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.599547 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] PATH_INFO: `/v3/projects/d36d1972e56d45509686359b8494d0bd/users/082d4bbf792b445faa335f1c3351f16e/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.615118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36d1972e56d45509686359b8494d0bd, user_id=082d4bbf792b445faa335f1c3351f16e, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.617879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.633754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1332/2657] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:21 2022] PUT /identity/v3/projects/d36d1972e56d45509686359b8494d0bd/users/082d4bbf792b445faa335f1c3351f16e/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:39:21.651358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e90c4-9f7e-4c8b-b2dc-a3503ff0a1cb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.651662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e90c4-9f7e-4c8b-b2dc-a3503ff0a1cb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.651969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e90c4-9f7e-4c8b-b2dc-a3503ff0a1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.681826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-936e90c4-9f7e-4c8b-b2dc-a3503ff0a1cb None None] MFA Rules not processed for user `082d4bbf792b445faa335f1c3351f16e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:21.683026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-936e90c4-9f7e-4c8b-b2dc-a3503ff0a1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:21.750225 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1326/2658] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:21.755349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.821955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fca17-2152-4150-ad7b-c21f75c9bedf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.827187 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.827798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.828081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.828404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.829048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.830986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.854674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:21.865436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1333/2659] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:21 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:21.866690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e90c4-9f7e-4c8b-b2dc-a3503ff0a1cb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.878005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e90c4-9f7e-4c8b-b2dc-a3503ff0a1cb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.938309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.938924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.939199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.939470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.940107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.942216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:21.966685 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1327/2660] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:21 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:21.972199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:21.983453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fae91c-e271-4043-9e0f-54b3c0c615ae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:21.988823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:21.989768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:21.989768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:21.989988 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:21.990624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:21.992285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.003035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1334/2661] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:22.007353 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:22.012844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:22.018751 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5f72d-0ecd-4ac4-8bcf-fa15594d350a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:22.029773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:22.030378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.030653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.030950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] PATH_INFO: `/v3/projects/bfae5288dbfe4a7392ec6b41c94da1ec/users/04cab6c39f42451f9d8cea0fe7976ea9/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.047499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfae5288dbfe4a7392ec6b41c94da1ec, user_id=04cab6c39f42451f9d8cea0fe7976ea9, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:22.050177 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.066194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1328/2662] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:22 2022] PUT /identity/v3/projects/bfae5288dbfe4a7392ec6b41c94da1ec/users/04cab6c39f42451f9d8cea0fe7976ea9/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:22.070439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:22.080949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24569f8b-cfae-4ac5-8f6b-06f184898554 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:22.143019 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c2393-355c-431c-a641-1cf90b05d526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:22.143694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c2393-355c-431c-a641-1cf90b05d526 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.143977 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c2393-355c-431c-a641-1cf90b05d526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.144289 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c2393-355c-431c-a641-1cf90b05d526 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.144928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c2393-355c-431c-a641-1cf90b05d526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:22.146883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c2393-355c-431c-a641-1cf90b05d526 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.157179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1335/2663] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:22.161546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:22.172055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661415d4-d6c1-4dea-b149-e20be22d645d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:22.176845 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:22.177491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.177764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.178031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] PATH_INFO: `/v3/projects/bfae5288dbfe4a7392ec6b41c94da1ec/users/04cab6c39f42451f9d8cea0fe7976ea9/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.179944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfae5288dbfe4a7392ec6b41c94da1ec, user_id=04cab6c39f42451f9d8cea0fe7976ea9, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:22.182599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.193334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1329/2664] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:22 2022] PUT /identity/v3/projects/bfae5288dbfe4a7392ec6b41c94da1ec/users/04cab6c39f42451f9d8cea0fe7976ea9/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:22.208177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d4fa6-260e-4d30-9de7-c2f21256dd4a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.208715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d4fa6-260e-4d30-9de7-c2f21256dd4a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.209016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d4fa6-260e-4d30-9de7-c2f21256dd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.240231 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-281d4fa6-260e-4d30-9de7-c2f21256dd4a None None] MFA Rules not processed for user `04cab6c39f42451f9d8cea0fe7976ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:22.241463 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-281d4fa6-260e-4d30-9de7-c2f21256dd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:22.314914 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1336/2665] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:22.327554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:22.329760 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:22.393923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c83a7c-c7aa-45a9-b2e0-59a2f1775011 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:22.400770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:22.401678 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.402067 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.402435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.416385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:22.417880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.429618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1330/2666] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:22.768998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d4fa6-260e-4d30-9de7-c2f21256dd4a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:22.783350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d4fa6-260e-4d30-9de7-c2f21256dd4a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:22.788795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cab6c39f42451f9d8cea0fe7976ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfae5288dbfe4a7392ec6b41c94da1ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:22.789417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.789722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.790000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.790651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:39:22.790960 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:22.792676 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.804971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1337/2667] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:39:22.809550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:22.819623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d066a-c68e-45df-bee2-90c8d22a2264 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:22.824453 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cab6c39f42451f9d8cea0fe7976ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfae5288dbfe4a7392ec6b41c94da1ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:22.825084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.825367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.825643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.826291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:22.828193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.858331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1331/2668] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:22 2022] POST /identity/v3/projects => generated 354 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:39:22.862508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:22.872957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2173d99-f9c0-4d5f-a223-ddca00412d3a tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:22.927398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:22.934369 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cab6c39f42451f9d8cea0fe7976ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfae5288dbfe4a7392ec6b41c94da1ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:22.934993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:22.935277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:22.935556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:22.936261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:39:22.936572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:22.938217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:22.951798 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1338/2669] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:39:22.990942 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:23.001538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937f160-2f68-43df-a561-c7c96454f780 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:23.006639 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cab6c39f42451f9d8cea0fe7976ea9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfae5288dbfe4a7392ec6b41c94da1ec', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:23.007273 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:23.007558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:23.007829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] PATH_INFO: `/v3/projects/c650b49b6bf149929f4a87c227874820` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:23.009059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c650b49b6bf149929f4a87c227874820)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:23.010972 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:23.069015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c650b49b6bf149929f4a87c227874820', 'request_id': 'req-063cc9a7-8666-42e0-b71c-a39cae7ca747'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.075282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c650b49b6bf149929f4a87c227874820', 'request_id': 'req-063cc9a7-8666-42e0-b71c-a39cae7ca747'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.082155 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c650b49b6bf149929f4a87c227874820', 'request_id': 'req-063cc9a7-8666-42e0-b71c-a39cae7ca747'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.100443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] The token cache is being invalidate because project c650b49b6bf149929f4a87c227874820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:23.100787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.101486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-063cc9a7-8666-42e0-b71c-a39cae7ca747 tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.103130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1332/2670] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:22 2022] DELETE /identity/v3/projects/c650b49b6bf149929f4a87c227874820 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:23.319778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:23.321739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:23.387516 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2cfaf-fb90-444a-b219-2cf95b27fd8f tempest-VolumeQuotaClassesTest-954252341 tempest-VolumeQuotaClassesTest-954252341-project-admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:23.447948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:23.448621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:23.448893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:23.449170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] PATH_INFO: `/v3/users/082d4bbf792b445faa335f1c3351f16e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:23.450347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=082d4bbf792b445faa335f1c3351f16e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:23.452302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:23.523807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082d4bbf792b445faa335f1c3351f16e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.533239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082d4bbf792b445faa335f1c3351f16e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.545457 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082d4bbf792b445faa335f1c3351f16e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.546109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082d4bbf792b445faa335f1c3351f16e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.553008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082d4bbf792b445faa335f1c3351f16e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:23.559660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1339/2671] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:23 2022] DELETE /identity/v3/users/082d4bbf792b445faa335f1c3351f16e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:23.564467 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:23.564938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:23.565023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:23.585172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:23.586408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:23.619531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:23.658304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1333/2672] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:23.671690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:23.674124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:23.697004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:23.738544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666701b9-a7b9-4453-ada1-bc7ccc8bdfd8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:23.742829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:23.743455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:23.743739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:23.744013 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:23.753784 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:23.755245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:23.764686 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1340/2673] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:24.039868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:24.053644 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:24.055716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a46dde-efd1-4027-a599-b7b830048948 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:24.061338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:24.062019 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:24.062298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:24.062565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] PATH_INFO: `/v3/projects/d36d1972e56d45509686359b8494d0bd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:24.063709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d36d1972e56d45509686359b8494d0bd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:24.065635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:24.122170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36d1972e56d45509686359b8494d0bd', 'request_id': 'req-49ae99a4-78b6-4f2d-9c19-da6009a8858b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.129730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36d1972e56d45509686359b8494d0bd', 'request_id': 'req-49ae99a4-78b6-4f2d-9c19-da6009a8858b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.136833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36d1972e56d45509686359b8494d0bd', 'request_id': 'req-49ae99a4-78b6-4f2d-9c19-da6009a8858b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.157126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] The token cache is being invalidate because project d36d1972e56d45509686359b8494d0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:24.157511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.158518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.160993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1334/2674] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:24 2022] DELETE /identity/v3/projects/d36d1972e56d45509686359b8494d0bd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:24.165926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:24.175936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05618a-49dd-4845-85ac-d8588817af2c service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:24.245073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:24.245704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:24.245979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:24.246243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] PATH_INFO: `/v3/users/04cab6c39f42451f9d8cea0fe7976ea9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:24.247404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04cab6c39f42451f9d8cea0fe7976ea9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:24.249377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:24.298048 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04cab6c39f42451f9d8cea0fe7976ea9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.306262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04cab6c39f42451f9d8cea0fe7976ea9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.318681 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04cab6c39f42451f9d8cea0fe7976ea9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.319314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04cab6c39f42451f9d8cea0fe7976ea9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.326118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-96bf72f2-1875-41d7-b29c-a7d02e60845d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04cab6c39f42451f9d8cea0fe7976ea9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.331969 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1341/2675] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:24 2022] DELETE /identity/v3/users/04cab6c39f42451f9d8cea0fe7976ea9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:24.606720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:24.608494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:24.623219 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae99a4-78b6-4f2d-9c19-da6009a8858b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:24.683402 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:24.684046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:24.684374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:24.684632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] PATH_INFO: `/v3/projects/bfae5288dbfe4a7392ec6b41c94da1ec` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:24.685804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfae5288dbfe4a7392ec6b41c94da1ec)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:24.687822 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:24.737447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfae5288dbfe4a7392ec6b41c94da1ec', 'request_id': 'req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.741454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:24.743561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfae5288dbfe4a7392ec6b41c94da1ec', 'request_id': 'req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.750904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfae5288dbfe4a7392ec6b41c94da1ec', 'request_id': 'req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.766371 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] The token cache is being invalidate because project bfae5288dbfe4a7392ec6b41c94da1ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:24.766665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.767434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d46d43ae-4aaf-4eda-8f94-a38bcf934b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:24.769108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1335/2676] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:24 2022] DELETE /identity/v3/projects/bfae5288dbfe4a7392ec6b41c94da1ec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:24.774496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a4b62-b083-49a0-ae1e-7d5c09cf144a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:24.774868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a4b62-b083-49a0-ae1e-7d5c09cf144a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:24.775222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a4b62-b083-49a0-ae1e-7d5c09cf144a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:24.797727 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-823a4b62-b083-49a0-ae1e-7d5c09cf144a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:24.798930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-823a4b62-b083-49a0-ae1e-7d5c09cf144a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:24.836483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:24.865620 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1342/2677] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:24.873516 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504bee-b8c6-459f-a5e9-ef14072328af None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:24.873841 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504bee-b8c6-459f-a5e9-ef14072328af None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:24.874120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504bee-b8c6-459f-a5e9-ef14072328af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:24.896412 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6f504bee-b8c6-459f-a5e9-ef14072328af None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:24.897833 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6f504bee-b8c6-459f-a5e9-ef14072328af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:24.906659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1336/2678] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:24.911069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a4b62-b083-49a0-ae1e-7d5c09cf144a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:24.921603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a4b62-b083-49a0-ae1e-7d5c09cf144a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:24.927687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:24.928351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:24.928630 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:24.928896 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:24.929548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:39:24.929852 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:24.931474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:24.943690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1343/2679] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:39:24.948079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f504bee-b8c6-459f-a5e9-ef14072328af None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:24.957783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f504bee-b8c6-459f-a5e9-ef14072328af None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:24.964501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:24.965101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:24.965373 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:24.965637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:24.966265 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:24.968186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:24.998486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1337/2680] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:24 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:25.002819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.013923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d2e50-0b81-4475-b32e-810613858756 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.078755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.079471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.079982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.080245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.080883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.082815 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.109658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1344/2681] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:25 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:25.114282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.125436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa006d-f5c8-4856-98a1-4624ef2443e7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.130978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.131808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.131935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.132249 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.132924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.134567 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.145393 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1338/2682] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:25.150112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.160952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94d66c-cff6-4c1e-b9a2-cdd01943ebe7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.166647 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeaae24-1b21-47cd-b807-f9a22283ed73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.167248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaae24-1b21-47cd-b807-f9a22283ed73 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.167531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaae24-1b21-47cd-b807-f9a22283ed73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.167803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeaae24-1b21-47cd-b807-f9a22283ed73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.168557 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeaae24-1b21-47cd-b807-f9a22283ed73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.170176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeaae24-1b21-47cd-b807-f9a22283ed73 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.181339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1345/2683] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:25.185515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.196227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54858d5e-cc8a-412c-95ea-9c01b81147f1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.201632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.202311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.202589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.202863 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] PATH_INFO: `/v3/projects/0ea564e566264777b38b3df3c38446ee/users/da6fda869ace43dbba0f510ceb1e6ed1/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.219001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea564e566264777b38b3df3c38446ee, user_id=da6fda869ace43dbba0f510ceb1e6ed1, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.221691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.236832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1339/2684] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:25 2022] PUT /identity/v3/projects/0ea564e566264777b38b3df3c38446ee/users/da6fda869ace43dbba0f510ceb1e6ed1/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:25.249144 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1037e9-577e-49e4-8d6e-7d10d283063e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.249428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1037e9-577e-49e4-8d6e-7d10d283063e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.249702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1037e9-577e-49e4-8d6e-7d10d283063e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.279475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9c1037e9-577e-49e4-8d6e-7d10d283063e None None] MFA Rules not processed for user `da6fda869ace43dbba0f510ceb1e6ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:25.280712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9c1037e9-577e-49e4-8d6e-7d10d283063e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:25.355860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1346/2685] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:25.361202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.428888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7286a88-0ff3-4122-b953-9f08c225259e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.434791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.435401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.435682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.435956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.436634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.438571 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.469747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1340/2686] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:25 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:25.474002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1037e9-577e-49e4-8d6e-7d10d283063e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.484987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1037e9-577e-49e4-8d6e-7d10d283063e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.517990 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:25.543484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.544101 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.544420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.544695 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.545331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.547293 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.569821 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1347/2687] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:25 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:25.574005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.584854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72433f-259c-4180-96a5-9d1ccec2a0fa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.590130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.590714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.591004 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.591304 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.591947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.593586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.604645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1341/2688] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:25.608697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.619636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef7cc-f12b-424d-8a3a-3fc34167ef45 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.625346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df108d-b97d-47c9-bd0f-2f57c77d57e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.625985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df108d-b97d-47c9-bd0f-2f57c77d57e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.626232 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df108d-b97d-47c9-bd0f-2f57c77d57e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.626503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df108d-b97d-47c9-bd0f-2f57c77d57e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.627136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df108d-b97d-47c9-bd0f-2f57c77d57e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.628790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df108d-b97d-47c9-bd0f-2f57c77d57e9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.644802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1348/2689] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:25.649481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.660303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9f07-f599-48e1-804e-ab43beb91524 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.666133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.666786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.667063 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.667333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] PATH_INFO: `/v3/projects/3f3aba0dfa8b42b98d22d762578baebf/users/cb0d645860634cd8aeb8939349182608/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.683573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f3aba0dfa8b42b98d22d762578baebf, user_id=cb0d645860634cd8aeb8939349182608, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.686262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.701502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1342/2690] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:25 2022] PUT /identity/v3/projects/3f3aba0dfa8b42b98d22d762578baebf/users/cb0d645860634cd8aeb8939349182608/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:25.712777 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d5fa-ea1d-4dac-8800-af716360bc9b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.713084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d5fa-ea1d-4dac-8800-af716360bc9b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.713415 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d5fa-ea1d-4dac-8800-af716360bc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.743158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7231d5fa-ea1d-4dac-8800-af716360bc9b None None] MFA Rules not processed for user `cb0d645860634cd8aeb8939349182608`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:25.744420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7231d5fa-ea1d-4dac-8800-af716360bc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:25.815799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1349/2691] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:25.820667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.853380 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:25.885917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9440a-9bc2-4917-8401-15e94c4201b9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:25.891076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:25.891679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:25.891955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:25.892257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:25.892881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:25.894814 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:25.923469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1343/2692] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:25 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:25.927689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231d5fa-ea1d-4dac-8800-af716360bc9b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:25.939225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231d5fa-ea1d-4dac-8800-af716360bc9b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:26.002772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:26.003384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:26.003661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:26.003926 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:26.004580 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:26.006633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:26.032473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1350/2693] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:25 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:26.037173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:26.053627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0deae-352a-45de-b7ed-f1d1ff460949 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:26.059410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:26.060008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:26.060317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:26.060579 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:26.061212 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:26.062826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:26.073826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1344/2694] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:26.077953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:26.088817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84545-b43b-416f-b4f9-a7142a3dc9b8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:26.094344 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:26.095002 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:26.095269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:26.095545 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] PATH_INFO: `/v3/projects/27f8745c576c488cb9ebff385e70340b/users/90e13e38ea854db7a848a9c3cb3760d1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:26.111037 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f8745c576c488cb9ebff385e70340b, user_id=90e13e38ea854db7a848a9c3cb3760d1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:26.114111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:26.128291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1351/2695] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:26 2022] PUT /identity/v3/projects/27f8745c576c488cb9ebff385e70340b/users/90e13e38ea854db7a848a9c3cb3760d1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:26.132661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:26.142798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6e012-5bbc-47f6-8d67-b083d5971189 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:26.200559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52802-14a0-459f-871b-e19b92314778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:26.201170 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52802-14a0-459f-871b-e19b92314778 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:26.201447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52802-14a0-459f-871b-e19b92314778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:26.201723 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52802-14a0-459f-871b-e19b92314778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:26.202346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d52802-14a0-459f-871b-e19b92314778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:26.203967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d52802-14a0-459f-871b-e19b92314778 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:26.214280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1345/2696] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:39:26.218847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:26.229989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e8428-ac2d-4a76-b41a-ae6072dd2cea admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:26.235688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:26.236397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:26.236659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:26.236931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] PATH_INFO: `/v3/projects/27f8745c576c488cb9ebff385e70340b/users/90e13e38ea854db7a848a9c3cb3760d1/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:26.239060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f8745c576c488cb9ebff385e70340b, user_id=90e13e38ea854db7a848a9c3cb3760d1, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:26.242167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:26.245532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:26.255181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1352/2697] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:26 2022] PUT /identity/v3/projects/27f8745c576c488cb9ebff385e70340b/users/90e13e38ea854db7a848a9c3cb3760d1/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:39:26.271337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37005-e424-4812-a87c-f3b5755d68b6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:26.271624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37005-e424-4812-a87c-f3b5755d68b6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:26.272045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a37005-e424-4812-a87c-f3b5755d68b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:26.303005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-02a37005-e424-4812-a87c-f3b5755d68b6 None None] MFA Rules not processed for user `90e13e38ea854db7a848a9c3cb3760d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:26.304561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-02a37005-e424-4812-a87c-f3b5755d68b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:26.380100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1346/2698] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:26.391112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:26.393050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:26.453641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5de1a6-cdfc-4e21-9c22-e773ac8fc7c0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:26.458319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:26.458950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:26.459225 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:26.459518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:26.469599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:26.471041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:26.480519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1353/2699] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:27.009819 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37005-e424-4812-a87c-f3b5755d68b6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:27.019535 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a37005-e424-4812-a87c-f3b5755d68b6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:27.023386 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:27.023992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:27.024306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:27.024577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:27.045491 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:27.083462 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:27.085200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:27.106242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1347/2700] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:31.277892 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:31.287429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d965b-49e6-4fe0-85ba-a613c0365a77 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:31.291387 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e07a8-a271-4f3f-b267-cf4ab6745957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:31.292020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e07a8-a271-4f3f-b267-cf4ab6745957 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:31.292315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e07a8-a271-4f3f-b267-cf4ab6745957 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:31.292585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e07a8-a271-4f3f-b267-cf4ab6745957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:31.350378 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e07a8-a271-4f3f-b267-cf4ab6745957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:31.352102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e07a8-a271-4f3f-b267-cf4ab6745957 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:31.361212 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1354/2701] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:33.559873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:33.623306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6432882-4ba3-46ae-b51f-5edee2c7a406 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:33.628026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:33.628679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:33.628954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:33.629220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] PATH_INFO: `/v3/users/da6fda869ace43dbba0f510ceb1e6ed1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:33.630329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da6fda869ace43dbba0f510ceb1e6ed1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:33.632374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:33.684626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da6fda869ace43dbba0f510ceb1e6ed1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:33.685427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da6fda869ace43dbba0f510ceb1e6ed1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:33.694712 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da6fda869ace43dbba0f510ceb1e6ed1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:33.709649 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da6fda869ace43dbba0f510ceb1e6ed1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:33.710383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da6fda869ace43dbba0f510ceb1e6ed1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:33.724957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1348/2702] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:33 2022] DELETE /identity/v3/users/da6fda869ace43dbba0f510ceb1e6ed1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:33.730054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:33.730521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:33.730632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:33.750627 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:33.751831 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:33.821114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1355/2703] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:33.831328 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:33.833021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:33.881474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:33.895031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed462cba-116d-412f-b040-364c1e5756f1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:33.899351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:33.899992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:33.900345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:33.900622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:33.910127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:33.911572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:33.921316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1349/2704] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:34.199935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:34.201433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:34.216005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bede93a-6b5a-44be-b34a-e2453fd8e816 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:34.221688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:34.222414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:34.222822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:34.222993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] PATH_INFO: `/v3/projects/0ea564e566264777b38b3df3c38446ee` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:34.224219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea564e566264777b38b3df3c38446ee)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:34.226194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:34.265140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:34.278194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea564e566264777b38b3df3c38446ee', 'request_id': 'req-714c3c29-eef2-424f-b4a7-4e882732c809'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.284809 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea564e566264777b38b3df3c38446ee', 'request_id': 'req-714c3c29-eef2-424f-b4a7-4e882732c809'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.290756 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea564e566264777b38b3df3c38446ee', 'request_id': 'req-714c3c29-eef2-424f-b4a7-4e882732c809'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.305304 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] The token cache is being invalidate because project 0ea564e566264777b38b3df3c38446ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:34.305590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.306223 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.307793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1356/2705] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:34 2022] DELETE /identity/v3/projects/0ea564e566264777b38b3df3c38446ee => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:34.311951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:34.321915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe99a8-2a7d-40f3-8e50-fcdced9a3c16 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:34.384570 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:34.385266 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:34.385573 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:34.385824 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] PATH_INFO: `/v3/users/cb0d645860634cd8aeb8939349182608` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:34.387077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb0d645860634cd8aeb8939349182608)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:34.389388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:34.451111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb0d645860634cd8aeb8939349182608'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.451838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb0d645860634cd8aeb8939349182608'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.463804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb0d645860634cd8aeb8939349182608'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.477931 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb0d645860634cd8aeb8939349182608'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.478711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb0d645860634cd8aeb8939349182608'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.502051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1350/2706] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:34 2022] DELETE /identity/v3/users/cb0d645860634cd8aeb8939349182608 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:34.772994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:34.774536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:34.787671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c3c29-eef2-424f-b4a7-4e882732c809 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:34.848752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:34.849400 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:34.849683 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:34.849954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] PATH_INFO: `/v3/projects/3f3aba0dfa8b42b98d22d762578baebf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:34.851118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f3aba0dfa8b42b98d22d762578baebf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:34.853103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:34.904136 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f3aba0dfa8b42b98d22d762578baebf', 'request_id': 'req-a653dc3a-4a12-4e53-b472-b3fc1eea4525'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.910590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f3aba0dfa8b42b98d22d762578baebf', 'request_id': 'req-a653dc3a-4a12-4e53-b472-b3fc1eea4525'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.917047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f3aba0dfa8b42b98d22d762578baebf', 'request_id': 'req-a653dc3a-4a12-4e53-b472-b3fc1eea4525'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.930983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] The token cache is being invalidate because project 3f3aba0dfa8b42b98d22d762578baebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:34.931275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.931959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:34.933613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1357/2707] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:34 2022] DELETE /identity/v3/projects/3f3aba0dfa8b42b98d22d762578baebf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:34.937696 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:34.939153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:34.952186 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624a2f6-19fb-423d-8701-ceaf77fcd661 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:35.011759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:35.012413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.012690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.012956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] PATH_INFO: `/v3/users/90e13e38ea854db7a848a9c3cb3760d1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.014121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e13e38ea854db7a848a9c3cb3760d1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:35.016039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:35.021365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:35.068770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e13e38ea854db7a848a9c3cb3760d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.069410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e13e38ea854db7a848a9c3cb3760d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.077562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e13e38ea854db7a848a9c3cb3760d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.090022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e13e38ea854db7a848a9c3cb3760d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.090758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-debcbd1e-ab8e-4cf6-ab90-8df017729702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e13e38ea854db7a848a9c3cb3760d1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.103691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1351/2708] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:34 2022] DELETE /identity/v3/users/90e13e38ea854db7a848a9c3cb3760d1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:35.403489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:35.405934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:35.424539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653dc3a-4a12-4e53-b472-b3fc1eea4525 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:35.476766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:35.488591 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:35.489553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.489807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.490081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] PATH_INFO: `/v3/projects/27f8745c576c488cb9ebff385e70340b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.491267 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f8745c576c488cb9ebff385e70340b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:35.493250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:35.546820 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f8745c576c488cb9ebff385e70340b', 'request_id': 'req-f8378462-0920-498c-9293-1caad96172c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.553211 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f8745c576c488cb9ebff385e70340b', 'request_id': 'req-f8378462-0920-498c-9293-1caad96172c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.559629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f8745c576c488cb9ebff385e70340b', 'request_id': 'req-f8378462-0920-498c-9293-1caad96172c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.574593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] The token cache is being invalidate because project 27f8745c576c488cb9ebff385e70340b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:35.574885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.575501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f8378462-0920-498c-9293-1caad96172c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:35.577113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1358/2709] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:35 2022] DELETE /identity/v3/projects/27f8745c576c488cb9ebff385e70340b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:35.583272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70b8ee-f1e6-42a7-9335-7d8b98f09cf8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.583556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70b8ee-f1e6-42a7-9335-7d8b98f09cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.583827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70b8ee-f1e6-42a7-9335-7d8b98f09cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.604493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ac70b8ee-f1e6-42a7-9335-7d8b98f09cf8 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:35.605646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ac70b8ee-f1e6-42a7-9335-7d8b98f09cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:35.682348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1352/2710] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:35.691003 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79108e-584d-4213-8c75-cc4acef9e81a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.691290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79108e-584d-4213-8c75-cc4acef9e81a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.691564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79108e-584d-4213-8c75-cc4acef9e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.714178 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6c79108e-584d-4213-8c75-cc4acef9e81a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:35.715467 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c79108e-584d-4213-8c75-cc4acef9e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:35.725323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1359/2711] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:35.729939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70b8ee-f1e6-42a7-9335-7d8b98f09cf8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:35.740168 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70b8ee-f1e6-42a7-9335-7d8b98f09cf8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:35.745278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:35.745879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.746153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.746418 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.747064 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:39:35.747360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:35.749018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:35.760718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1353/2712] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:39:35.765445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79108e-584d-4213-8c75-cc4acef9e81a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:35.774832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79108e-584d-4213-8c75-cc4acef9e81a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:35.779623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:35.780380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.780659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.780929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.781568 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:35.783985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:35.814950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1360/2713] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:35 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:35.819379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:35.829991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc01f37-7d5e-4797-ad2f-17cabca2e88b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:35.889407 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:35.890008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.890287 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.890559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.891188 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:35.893152 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:35.912536 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:35.917710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1354/2714] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:35 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:35.922234 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:35.935380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a9534-5ba5-49ea-b0f6-9ab00325ce3d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:35.940599 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:35.941213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.941493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.941763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.942404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:35.944026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:35.954655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1361/2715] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:35.958807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:35.969764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aafed6-14c3-4048-a393-47a1c4e05b5f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:35.974620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5030288-b138-4251-9865-5b2b01c7dfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:35.975217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5030288-b138-4251-9865-5b2b01c7dfed admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:35.975493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5030288-b138-4251-9865-5b2b01c7dfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:35.975764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5030288-b138-4251-9865-5b2b01c7dfed admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:35.976433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5030288-b138-4251-9865-5b2b01c7dfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:35.978027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5030288-b138-4251-9865-5b2b01c7dfed admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:35.988710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1355/2716] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:35.992725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.002830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dea53-1701-4750-bc23-a24519a0ea3c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.007835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.008519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.008797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.009068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] PATH_INFO: `/v3/projects/557de62569f14f479c872cf3f86e50f1/users/890d64db41034ceba20a45c517f76a26/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.024574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557de62569f14f479c872cf3f86e50f1, user_id=890d64db41034ceba20a45c517f76a26, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.027368 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.042285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1362/2717] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:35 2022] PUT /identity/v3/projects/557de62569f14f479c872cf3f86e50f1/users/890d64db41034ceba20a45c517f76a26/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:36.053866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a3aa8-e253-4ebf-8ea0-97f4980e00e7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.054150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a3aa8-e253-4ebf-8ea0-97f4980e00e7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.054447 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a3aa8-e253-4ebf-8ea0-97f4980e00e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.082835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d03a3aa8-e253-4ebf-8ea0-97f4980e00e7 None None] MFA Rules not processed for user `890d64db41034ceba20a45c517f76a26`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:36.084094 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d03a3aa8-e253-4ebf-8ea0-97f4980e00e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:36.150256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1356/2718] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:36.155030 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.219307 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80851405-356f-4ea3-bdf4-c66e2a195db3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.224417 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.225028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.225302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.225546 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.226196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.228107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.264676 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1363/2719] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:39:36 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:36.268801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a3aa8-e253-4ebf-8ea0-97f4980e00e7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.280109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a3aa8-e253-4ebf-8ea0-97f4980e00e7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.336329 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.337280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.337559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.337818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.338489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.340437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.363650 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1357/2720] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:39:36 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:39:36.367818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.374255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:36.381829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd2c28-57cb-41c7-863b-d14b9072a6c4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.387791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.388679 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.389006 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.389469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.390130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.391766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.403277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1364/2721] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:36.407705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.417980 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd804f01-a69f-4c71-a4de-1f7a79501737 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.423303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.423963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.424299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.424577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] PATH_INFO: `/v3/projects/6872e0bcdf1743cfb0a7a8e29c97a3f9/users/f04f33e531cc42b6add3ae8c5d4a7deb/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.440295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6872e0bcdf1743cfb0a7a8e29c97a3f9, user_id=f04f33e531cc42b6add3ae8c5d4a7deb, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.442969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.458842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1358/2722] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:36 2022] PUT /identity/v3/projects/6872e0bcdf1743cfb0a7a8e29c97a3f9/users/f04f33e531cc42b6add3ae8c5d4a7deb/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:36.463465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.473712 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e2af3-85ee-4250-8487-d0a7323a4a98 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.532553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d12e2c-a597-4d2b-97c8-b41b40fc3593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.533057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d12e2c-a597-4d2b-97c8-b41b40fc3593 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.533339 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d12e2c-a597-4d2b-97c8-b41b40fc3593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.533613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d12e2c-a597-4d2b-97c8-b41b40fc3593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.534245 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d12e2c-a597-4d2b-97c8-b41b40fc3593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.535881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d12e2c-a597-4d2b-97c8-b41b40fc3593 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.546823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1365/2723] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:39:36.551141 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.561646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4b4a2-2d3e-4eb2-9f2d-088396466f0e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.566600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.567476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.567761 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.568034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] PATH_INFO: `/v3/projects/6872e0bcdf1743cfb0a7a8e29c97a3f9/users/f04f33e531cc42b6add3ae8c5d4a7deb/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.570072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6872e0bcdf1743cfb0a7a8e29c97a3f9, user_id=f04f33e531cc42b6add3ae8c5d4a7deb, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.572879 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.583652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1359/2724] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:39:36 2022] PUT /identity/v3/projects/6872e0bcdf1743cfb0a7a8e29c97a3f9/users/f04f33e531cc42b6add3ae8c5d4a7deb/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:36.595732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cc88a-f977-4661-afe4-e8ff133ab207 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.596022 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cc88a-f977-4661-afe4-e8ff133ab207 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.596323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0cc88a-f977-4661-afe4-e8ff133ab207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.625072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0b0cc88a-f977-4661-afe4-e8ff133ab207 None None] MFA Rules not processed for user `f04f33e531cc42b6add3ae8c5d4a7deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:36.626331 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0b0cc88a-f977-4661-afe4-e8ff133ab207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:36.696858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1366/2725] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:36.709671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.711546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:36.747959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:36.773754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e583d55-15ce-40f7-9afd-b231509edb2d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.778023 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.778643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.778922 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.779193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.788667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.790062 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:36.799982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1360/2726] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:36.903192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cc88a-f977-4661-afe4-e8ff133ab207 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:36.913259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0cc88a-f977-4661-afe4-e8ff133ab207 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:36.917778 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c30f3-e0cd-4286-9bf8-3ec31464bc4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:36.918395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c30f3-e0cd-4286-9bf8-3ec31464bc4c service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:36.918677 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c30f3-e0cd-4286-9bf8-3ec31464bc4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:36.918946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c30f3-e0cd-4286-9bf8-3ec31464bc4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:36.989104 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c30f3-e0cd-4286-9bf8-3ec31464bc4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:36.990747 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904c30f3-e0cd-4286-9bf8-3ec31464bc4c service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:37.000550 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1367/2727] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:37.324926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:37.392479 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382df34-2a7c-492d-b369-731145c06f14 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:37.397306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:37.397949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:37.398230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:37.398525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] PATH_INFO: `/v3/users/890d64db41034ceba20a45c517f76a26` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:37.399673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=890d64db41034ceba20a45c517f76a26)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:37.401857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:37.456856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '890d64db41034ceba20a45c517f76a26'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:37.457503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '890d64db41034ceba20a45c517f76a26'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:37.466316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '890d64db41034ceba20a45c517f76a26'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:37.478562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '890d64db41034ceba20a45c517f76a26'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:37.479227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '890d64db41034ceba20a45c517f76a26'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:37.491080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1361/2728] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:37 2022] DELETE /identity/v3/users/890d64db41034ceba20a45c517f76a26 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:37.496059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:37.496362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:37.496651 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:37.517973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:39:37.519222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:37.525502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:37.588429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1368/2729] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:39:37.598835 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:37.600330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:37.666708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34e681-599c-45f1-ab6e-5d4646c6d70a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:37.670501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:37.671116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:37.671392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:37.671654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:37.678945 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:37.681700 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:37.683562 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:37.693827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1362/2730] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:39:37.981657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:37.983329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:37.996579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec070e-d7db-4ecd-a787-07640c8087b6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:38.002207 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:38.002858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:38.003126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:38.003404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] PATH_INFO: `/v3/projects/557de62569f14f479c872cf3f86e50f1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:38.004663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=557de62569f14f479c872cf3f86e50f1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:38.006610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:38.055600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557de62569f14f479c872cf3f86e50f1', 'request_id': 'req-c66791c9-47f4-4c41-a972-c897138ca563'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.062026 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557de62569f14f479c872cf3f86e50f1', 'request_id': 'req-c66791c9-47f4-4c41-a972-c897138ca563'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.067589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557de62569f14f479c872cf3f86e50f1', 'request_id': 'req-c66791c9-47f4-4c41-a972-c897138ca563'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.081403 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] The token cache is being invalidate because project 557de62569f14f479c872cf3f86e50f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:38.081688 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.082295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.083782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1369/2731] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:37 2022] DELETE /identity/v3/projects/557de62569f14f479c872cf3f86e50f1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:38.088515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:38.098443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef46951-46d1-4c11-9e4f-daf8b197063d service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:38.159003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:38.159643 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:38.159924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:38.160193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] PATH_INFO: `/v3/users/f04f33e531cc42b6add3ae8c5d4a7deb` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:38.161368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04f33e531cc42b6add3ae8c5d4a7deb)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:38.163344 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:38.216668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04f33e531cc42b6add3ae8c5d4a7deb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.217316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04f33e531cc42b6add3ae8c5d4a7deb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.226291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04f33e531cc42b6add3ae8c5d4a7deb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.238736 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04f33e531cc42b6add3ae8c5d4a7deb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.239451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04f33e531cc42b6add3ae8c5d4a7deb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.251932 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1363/2732] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:39:38 2022] DELETE /identity/v3/users/f04f33e531cc42b6add3ae8c5d4a7deb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:38.543206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:39:38.544814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:39:38.559121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66791c9-47f4-4c41-a972-c897138ca563 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:39:38.631996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:39:38.632611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:39:38.632909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:39:38.633176 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] PATH_INFO: `/v3/projects/6872e0bcdf1743cfb0a7a8e29c97a3f9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:39:38.634348 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6872e0bcdf1743cfb0a7a8e29c97a3f9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:39:38.636506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:39:38.677061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:38.687085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6872e0bcdf1743cfb0a7a8e29c97a3f9', 'request_id': 'req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.693863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6872e0bcdf1743cfb0a7a8e29c97a3f9', 'request_id': 'req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.700458 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6872e0bcdf1743cfb0a7a8e29c97a3f9', 'request_id': 'req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.716490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] The token cache is being invalidate because project 6872e0bcdf1743cfb0a7a8e29c97a3f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:39:38.716779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.717465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:39:38.718945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1370/2733] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:39:38 2022] DELETE /identity/v3/projects/6872e0bcdf1743cfb0a7a8e29c97a3f9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:39:48.248855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:39:48.713299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:20.767572 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:20.769330 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:20.832440 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ccea4-8201-481e-8209-5073ba9af144 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:20.886621 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:20.887246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:20.887531 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:20.887806 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:20.890525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:20.953450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:20.954808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:20.964790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1364/2734] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:40:24.164810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:24.166544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:24.181056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeef45e-8c9f-41a4-a9a4-c735ee9decfd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:24.186753 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:24.187935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:24.188249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:24.188524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] PATH_INFO: `/v3/users/2b81fb00b1b545308d01f74a55b7e77b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:24.189787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b81fb00b1b545308d01f74a55b7e77b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:24.191792 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:24.256102 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b81fb00b1b545308d01f74a55b7e77b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.265979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b81fb00b1b545308d01f74a55b7e77b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.279877 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b81fb00b1b545308d01f74a55b7e77b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.280510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b81fb00b1b545308d01f74a55b7e77b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.287840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0e900c8d-567c-4c70-b904-e51183e4b996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b81fb00b1b545308d01f74a55b7e77b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.293984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1371/2735] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:40:24 2022] DELETE /identity/v3/users/2b81fb00b1b545308d01f74a55b7e77b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:24.487052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:24.497430 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a4c55-1671-457d-81a0-8dba553cf969 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:24.560811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:24.561454 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:24.561741 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:24.562466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] PATH_INFO: `/v3/projects/c6ed4f1f1246408099252ebea491832c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:24.563493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ed4f1f1246408099252ebea491832c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:24.565375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:24.573776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:24.620776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ed4f1f1246408099252ebea491832c', 'request_id': 'req-626e8c84-b710-443b-b6d1-8fd41b0b5579'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.627054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ed4f1f1246408099252ebea491832c', 'request_id': 'req-626e8c84-b710-443b-b6d1-8fd41b0b5579'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.633855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ed4f1f1246408099252ebea491832c', 'request_id': 'req-626e8c84-b710-443b-b6d1-8fd41b0b5579'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.649390 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] The token cache is being invalidate because project c6ed4f1f1246408099252ebea491832c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:40:24.649680 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.650336 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-626e8c84-b710-443b-b6d1-8fd41b0b5579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:24.651889 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1365/2736] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:40:24 2022] DELETE /identity/v3/projects/c6ed4f1f1246408099252ebea491832c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:24.658318 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1847875-c408-4d55-9295-4245aad4bf50 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:24.658649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1847875-c408-4d55-9295-4245aad4bf50 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:24.658923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1847875-c408-4d55-9295-4245aad4bf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:24.680158 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e1847875-c408-4d55-9295-4245aad4bf50 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:24.681388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e1847875-c408-4d55-9295-4245aad4bf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:24.746257 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1372/2737] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:40:24.756046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a16ddb-2b76-4992-91e2-428fb4b84367 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:24.756368 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a16ddb-2b76-4992-91e2-428fb4b84367 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:24.756646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a16ddb-2b76-4992-91e2-428fb4b84367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:24.779137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-04a16ddb-2b76-4992-91e2-428fb4b84367 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:24.780322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-04a16ddb-2b76-4992-91e2-428fb4b84367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:24.790869 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1366/2738] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:24.795676 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1847875-c408-4d55-9295-4245aad4bf50 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:24.807385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1847875-c408-4d55-9295-4245aad4bf50 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:24.813642 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:24.814253 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:24.814536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:24.814802 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:24.815488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:40:24.815787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:24.817450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:24.830738 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1373/2739] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:40:24.835354 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a16ddb-2b76-4992-91e2-428fb4b84367 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:24.845752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a16ddb-2b76-4992-91e2-428fb4b84367 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:24.851331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:24.852006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:24.852326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:24.852603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:24.853239 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:24.855164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:24.888450 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1367/2740] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:40:24 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:40:24.893383 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:24.903919 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8796042-7842-485e-a75e-80792494e1fc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:24.966068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:24.966682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:24.966956 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:24.967218 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:24.967843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:24.970449 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:24.978539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:24.998986 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1374/2741] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:40:24 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:40:25.000778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.011615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6154fa9-8cde-4b08-966d-a6ec92907bad admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.016818 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.017423 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.017704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.017944 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.018620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.020303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.030349 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1368/2742] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:25.035629 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.046185 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480bb7e-b0a5-440b-b281-a51f15843451 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.052093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de0fc4-94fc-4387-b6d5-4a4587e85799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.052735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de0fc4-94fc-4387-b6d5-4a4587e85799 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.053000 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de0fc4-94fc-4387-b6d5-4a4587e85799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.053323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de0fc4-94fc-4387-b6d5-4a4587e85799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.053958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de0fc4-94fc-4387-b6d5-4a4587e85799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.055573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de0fc4-94fc-4387-b6d5-4a4587e85799 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.066638 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1375/2743] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:25.071103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.082286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f278b41-f9c2-4e0b-b64f-568f8d6c3fc1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.087862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.088559 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.088832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.089103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] PATH_INFO: `/v3/projects/27bb493a94f040bdbe2fd212f53e1153/users/0b65bc03c3bc497a92f53432504054ee/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.106893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27bb493a94f040bdbe2fd212f53e1153, user_id=0b65bc03c3bc497a92f53432504054ee, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.110359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.126662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1369/2744] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:40:25 2022] PUT /identity/v3/projects/27bb493a94f040bdbe2fd212f53e1153/users/0b65bc03c3bc497a92f53432504054ee/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:25.137364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83549-faf1-41ea-8500-1f5650f9c713 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.137653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83549-faf1-41ea-8500-1f5650f9c713 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.137927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d83549-faf1-41ea-8500-1f5650f9c713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.168408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-31d83549-faf1-41ea-8500-1f5650f9c713 None None] MFA Rules not processed for user `0b65bc03c3bc497a92f53432504054ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:25.169623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-31d83549-faf1-41ea-8500-1f5650f9c713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:25.246450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1376/2745] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:25 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:25.251338 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.332332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa1da1-aa5a-4668-8417-337cfa06e7d5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.337926 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.338528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.338802 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.339072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.339694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.341619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.370797 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1370/2746] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:40:25 2022] POST /identity/v3/projects => generated 403 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:40:25.377098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83549-faf1-41ea-8500-1f5650f9c713 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.389595 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d83549-faf1-41ea-8500-1f5650f9c713 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.584541 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.585067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.585341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.585614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.586241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.588555 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.615519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1377/2747] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:40:25 2022] POST /identity/v3/users => generated 308 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:40:25.618970 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.631055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305b436-eec2-4740-93dc-d7da9f10cdf8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.635873 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.636501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.636779 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.637051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.637693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.639309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.650984 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1371/2748] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:25.656105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.666931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f51a-4e3d-4b20-965e-0d2390c0c6f7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.673113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.673810 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.674090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.674357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] PATH_INFO: `/v3/projects/589235580a9b4a4ab261e7d92f419971/users/5a6cf01b6b4c444788f905cc1b066a7a/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.689872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=589235580a9b4a4ab261e7d92f419971, user_id=5a6cf01b6b4c444788f905cc1b066a7a, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.692618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.707917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1378/2749] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:40:25 2022] PUT /identity/v3/projects/589235580a9b4a4ab261e7d92f419971/users/5a6cf01b6b4c444788f905cc1b066a7a/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:25.712282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.722546 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cdad1-2bce-4003-9d4e-1897b9179aff admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.726263 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:25.784645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d9fd68-99e2-4c64-adf6-fb4a835c2b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.785244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9fd68-99e2-4c64-adf6-fb4a835c2b73 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.785520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9fd68-99e2-4c64-adf6-fb4a835c2b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.785790 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9fd68-99e2-4c64-adf6-fb4a835c2b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.786424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9fd68-99e2-4c64-adf6-fb4a835c2b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.788047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d9fd68-99e2-4c64-adf6-fb4a835c2b73 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.798360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1372/2750] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:25.802613 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.813170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffafbd9-f4bb-4a9c-9d03-86f557355856 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:25.818008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:25.818655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.818927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.819195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] PATH_INFO: `/v3/projects/589235580a9b4a4ab261e7d92f419971/users/5a6cf01b6b4c444788f905cc1b066a7a/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.821167 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] RBAC: Authorizing `identity:create_grant(project_id=589235580a9b4a4ab261e7d92f419971, user_id=5a6cf01b6b4c444788f905cc1b066a7a, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:25.823811 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:25.834640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1379/2751] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:40:25 2022] PUT /identity/v3/projects/589235580a9b4a4ab261e7d92f419971/users/5a6cf01b6b4c444788f905cc1b066a7a/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:25.845267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed41b0-2815-42c5-8bed-93659b5c5f1b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:25.845552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed41b0-2815-42c5-8bed-93659b5c5f1b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:25.845829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed41b0-2815-42c5-8bed-93659b5c5f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:25.875844 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-30ed41b0-2815-42c5-8bed-93659b5c5f1b None None] MFA Rules not processed for user `5a6cf01b6b4c444788f905cc1b066a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:25.877077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-30ed41b0-2815-42c5-8bed-93659b5c5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:25.954179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1373/2752] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:25 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:25.965292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:25.967146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:25.990117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:26.032586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56370034-c9ea-4b87-a2ac-5d150343cbae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:26.036700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f560d-dfbe-48f0-9924-c11cc79dbb6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:26.037310 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f560d-dfbe-48f0-9924-c11cc79dbb6d service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:26.037590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f560d-dfbe-48f0-9924-c11cc79dbb6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:26.037860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f560d-dfbe-48f0-9924-c11cc79dbb6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:26.047265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f560d-dfbe-48f0-9924-c11cc79dbb6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:26.048755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f560d-dfbe-48f0-9924-c11cc79dbb6d service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:26.058196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1380/2753] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:40:25 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:40:26.218929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed41b0-2815-42c5-8bed-93659b5c5f1b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:26.289768 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed41b0-2815-42c5-8bed-93659b5c5f1b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:26.294952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:26.295615 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:26.295894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:26.296193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] PATH_INFO: `/v3/users/0b65bc03c3bc497a92f53432504054ee` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:26.297501 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b65bc03c3bc497a92f53432504054ee)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:26.299438 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:26.340103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:26.354849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b65bc03c3bc497a92f53432504054ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.355487 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b65bc03c3bc497a92f53432504054ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.365956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b65bc03c3bc497a92f53432504054ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.382715 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b65bc03c3bc497a92f53432504054ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.383469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b65bc03c3bc497a92f53432504054ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.397437 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1374/2754] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:40:26 2022] DELETE /identity/v3/users/0b65bc03c3bc497a92f53432504054ee => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:26.402570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:26.403126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:26.403126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:26.424473 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:26.425690 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:26.493583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1381/2755] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:26.506315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:26.507978 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:26.575149 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9bfa3-5ab4-4c62-a36c-647cb5a7c984 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:26.579664 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:26.580432 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:26.580686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:26.580989 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:26.591503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:26.592918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:26.602948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1375/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:40:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:40:26.886985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:26.888585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:26.901585 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca41db-31e8-4344-8f0e-01f558869054 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:26.907237 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:26.907873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:26.908161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:26.908440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] PATH_INFO: `/v3/projects/27bb493a94f040bdbe2fd212f53e1153` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:26.909691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27bb493a94f040bdbe2fd212f53e1153)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:26.911687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:26.970781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27bb493a94f040bdbe2fd212f53e1153', 'request_id': 'req-51019fc6-7b34-424e-9845-6cfc1b014354'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.977130 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27bb493a94f040bdbe2fd212f53e1153', 'request_id': 'req-51019fc6-7b34-424e-9845-6cfc1b014354'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.982853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27bb493a94f040bdbe2fd212f53e1153', 'request_id': 'req-51019fc6-7b34-424e-9845-6cfc1b014354'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.998194 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] The token cache is being invalidate because project 27bb493a94f040bdbe2fd212f53e1153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:40:26.998475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:26.999087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.001074 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1382/2757] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:40:26 2022] DELETE /identity/v3/projects/27bb493a94f040bdbe2fd212f53e1153 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:27.005193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:27.015288 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbcfb8-e205-4063-a09f-811cb392641b service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:27.074510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:27.075145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.075422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.075693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] PATH_INFO: `/v3/users/5a6cf01b6b4c444788f905cc1b066a7a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.076866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a6cf01b6b4c444788f905cc1b066a7a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:27.079022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:27.133939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a6cf01b6b4c444788f905cc1b066a7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.134554 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a6cf01b6b4c444788f905cc1b066a7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.138192 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:27.146659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a6cf01b6b4c444788f905cc1b066a7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.160929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a6cf01b6b4c444788f905cc1b066a7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.161628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-218f6bee-f906-48b1-9ed1-2cc1281131d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a6cf01b6b4c444788f905cc1b066a7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.174011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1376/2758] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:40:27 2022] DELETE /identity/v3/users/5a6cf01b6b4c444788f905cc1b066a7a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:27.464252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:27.465780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:27.481012 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51019fc6-7b34-424e-9845-6cfc1b014354 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:27.527512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:27.543299 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:27.543940 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.544255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.544529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] PATH_INFO: `/v3/projects/589235580a9b4a4ab261e7d92f419971` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.545632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=589235580a9b4a4ab261e7d92f419971)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:27.548100 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:27.596324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589235580a9b4a4ab261e7d92f419971', 'request_id': 'req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.603044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589235580a9b4a4ab261e7d92f419971', 'request_id': 'req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.608636 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589235580a9b4a4ab261e7d92f419971', 'request_id': 'req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.624046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] The token cache is being invalidate because project 589235580a9b4a4ab261e7d92f419971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:40:27.624483 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.625631 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1858a2a0-d3de-4864-b2be-e7bbff79c4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:27.627603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1383/2759] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:40:27 2022] DELETE /identity/v3/projects/589235580a9b4a4ab261e7d92f419971 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:27.632706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75e464-3733-4b17-8f95-0a7a7271d850 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.632995 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75e464-3733-4b17-8f95-0a7a7271d850 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.633272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75e464-3733-4b17-8f95-0a7a7271d850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.654910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ac75e464-3733-4b17-8f95-0a7a7271d850 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:27.656058 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ac75e464-3733-4b17-8f95-0a7a7271d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:27.720433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1377/2760] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:27.729291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6183-267c-4e49-a819-2731b0439e17 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.729577 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6183-267c-4e49-a819-2731b0439e17 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.729853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb6183-267c-4e49-a819-2731b0439e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.751224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-beeb6183-267c-4e49-a819-2731b0439e17 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:27.752466 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-beeb6183-267c-4e49-a819-2731b0439e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:27.761161 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1384/2761] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:27.765809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75e464-3733-4b17-8f95-0a7a7271d850 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:27.775992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75e464-3733-4b17-8f95-0a7a7271d850 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:27.781293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:27.781894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.782175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.782445 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.783090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:40:27.783383 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:27.796538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:27.798895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1378/2762] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:40:27.802928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb6183-267c-4e49-a819-2731b0439e17 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:27.819748 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb6183-267c-4e49-a819-2731b0439e17 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:27.824719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:27.825330 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.825605 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.825880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.826520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:27.829053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:27.862410 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1385/2763] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:40:27 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:40:27.867092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:27.878471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748b6e-6c93-47b8-b956-65aa2f105ed8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:27.944237 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:27.944762 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.945032 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.945306 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.945949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:27.947860 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:27.972892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1379/2764] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:40:27 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:40:27.977189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:27.988453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007e666-ccdc-4fd4-b48a-b634797ff3d6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:27.994172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:27.994760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:27.995033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:27.995300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:27.995933 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:27.997598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:28.008552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1386/2765] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:28.012480 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:28.023377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3032-e7e6-4288-b879-651535b512b2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:28.028250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94024dce-a98c-41b7-8d0f-04a9929ca68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:28.028842 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94024dce-a98c-41b7-8d0f-04a9929ca68c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:28.029114 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94024dce-a98c-41b7-8d0f-04a9929ca68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:28.029381 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94024dce-a98c-41b7-8d0f-04a9929ca68c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:28.030014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94024dce-a98c-41b7-8d0f-04a9929ca68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:28.031763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94024dce-a98c-41b7-8d0f-04a9929ca68c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:28.041801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1380/2766] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:28.045822 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:28.056250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee51ea3-ff1d-442c-9f9b-3eff356d7c2f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:28.061407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:28.062119 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:28.062493 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:28.063248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] PATH_INFO: `/v3/projects/d979398fac854b3a8cf947c98a690582/users/8fb3719aa3a440e5a8b619a67429c61f/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:28.077907 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d979398fac854b3a8cf947c98a690582, user_id=8fb3719aa3a440e5a8b619a67429c61f, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:28.080801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:28.097159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1387/2767] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:40:28 2022] PUT /identity/v3/projects/d979398fac854b3a8cf947c98a690582/users/8fb3719aa3a440e5a8b619a67429c61f/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:28.111600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2803e-7a41-4050-a59f-7c43f01c97d8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:28.111882 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2803e-7a41-4050-a59f-7c43f01c97d8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:28.112172 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2803e-7a41-4050-a59f-7c43f01c97d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:28.142769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c9f2803e-7a41-4050-a59f-7c43f01c97d8 None None] MFA Rules not processed for user `8fb3719aa3a440e5a8b619a67429c61f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:28.143986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c9f2803e-7a41-4050-a59f-7c43f01c97d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:28.170222 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:28.223027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1381/2768] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:28.234221 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:28.236490 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:28.304704 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c49c2-3787-4df0-8ab5-e7fbed742b63 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:28.308763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05afed-c736-4d52-895a-22b35128baa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:28.309377 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05afed-c736-4d52-895a-22b35128baa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:28.309655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05afed-c736-4d52-895a-22b35128baa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:28.309929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05afed-c736-4d52-895a-22b35128baa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:28.319979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05afed-c736-4d52-895a-22b35128baa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:28.321478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05afed-c736-4d52-895a-22b35128baa4 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:28.331205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1388/2769] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:40:28.350953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2803e-7a41-4050-a59f-7c43f01c97d8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:28.427499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2803e-7a41-4050-a59f-7c43f01c97d8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:28.434022 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:28.434660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:28.434935 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:28.435202 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] PATH_INFO: `/v3/users/8fb3719aa3a440e5a8b619a67429c61f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:28.436603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb3719aa3a440e5a8b619a67429c61f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:28.438634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:28.494481 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb3719aa3a440e5a8b619a67429c61f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:28.495075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb3719aa3a440e5a8b619a67429c61f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:28.506825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb3719aa3a440e5a8b619a67429c61f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:28.518906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb3719aa3a440e5a8b619a67429c61f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:28.519534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb3719aa3a440e5a8b619a67429c61f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:28.531665 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1382/2770] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:40:28 2022] DELETE /identity/v3/users/8fb3719aa3a440e5a8b619a67429c61f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:28.536420 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:28.536883 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:28.536983 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:28.557586 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:28.558789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:28.569477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:28.629112 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1389/2771] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:28.639119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:28.640666 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:28.688795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:28.701314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7ad58-2d96-4b7d-b2b9-232c42e5a986 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:28.705460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58590eec-1e7d-4a4f-b2d1-29c46f637b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:28.706065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58590eec-1e7d-4a4f-b2d1-29c46f637b17 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:28.706339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58590eec-1e7d-4a4f-b2d1-29c46f637b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:28.706607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58590eec-1e7d-4a4f-b2d1-29c46f637b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:28.715967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58590eec-1e7d-4a4f-b2d1-29c46f637b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:28.717427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58590eec-1e7d-4a4f-b2d1-29c46f637b17 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:28.726899 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1383/2772] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:40:29.002198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.003694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:29.023722 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b6355-9bed-4cdf-9779-a90b395c2248 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.028346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.028996 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.029275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.029552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] PATH_INFO: `/v3/projects/d979398fac854b3a8cf947c98a690582` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.030696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d979398fac854b3a8cf947c98a690582)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.032847 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.081150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd979398fac854b3a8cf947c98a690582', 'request_id': 'req-4ea26022-03bd-422e-8164-dc1a8843396d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:29.087684 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd979398fac854b3a8cf947c98a690582', 'request_id': 'req-4ea26022-03bd-422e-8164-dc1a8843396d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:29.093751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd979398fac854b3a8cf947c98a690582', 'request_id': 'req-4ea26022-03bd-422e-8164-dc1a8843396d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:29.108448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] The token cache is being invalidate because project d979398fac854b3a8cf947c98a690582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:40:29.108735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:29.109345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ea26022-03bd-422e-8164-dc1a8843396d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:40:29.111456 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1390/2773] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:40:29 2022] DELETE /identity/v3/projects/d979398fac854b3a8cf947c98a690582 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:29.116598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010ea0c-7c21-4f3d-ad17-615b6b39f255 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.117099 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010ea0c-7c21-4f3d-ad17-615b6b39f255 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.117200 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010ea0c-7c21-4f3d-ad17-615b6b39f255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.137594 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9010ea0c-7c21-4f3d-ad17-615b6b39f255 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:29.138789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9010ea0c-7c21-4f3d-ad17-615b6b39f255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:29.202316 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1384/2774] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:40:29.214143 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f813-ee4b-4c5e-a62e-07eb5a724bca None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.214506 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f813-ee4b-4c5e-a62e-07eb5a724bca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.214859 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f813-ee4b-4c5e-a62e-07eb5a724bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.241998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6cf3f813-ee4b-4c5e-a62e-07eb5a724bca None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:29.243574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6cf3f813-ee4b-4c5e-a62e-07eb5a724bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:29.254298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1391/2775] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:29.258698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010ea0c-7c21-4f3d-ad17-615b6b39f255 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.268854 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010ea0c-7c21-4f3d-ad17-615b6b39f255 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.281503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.282256 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.282534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.282784 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.283568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:40:29.283866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.285525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.300253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1385/2776] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:40:29.305363 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3f813-ee4b-4c5e-a62e-07eb5a724bca None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.316800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3f813-ee4b-4c5e-a62e-07eb5a724bca None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.323732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.324570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.324930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.325291 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.326029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.328107 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.359884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1392/2777] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:40:29 2022] POST /identity/v3/projects => generated 369 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:40:29.364309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.374969 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ec957-700d-4fc7-9f8d-5764b3a7950c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.436482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.437102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.437379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.437645 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.439504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.440815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.478703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1386/2778] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:40:29 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:40:29.484618 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.503477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afaeb2-4b8c-4b2e-b3fe-daad25b505a8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.510381 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.510999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.511278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.511551 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.512179 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.513827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.525370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1393/2779] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:29.531319 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.544385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b1af3-f1f0-4c3c-8051-061fd23d5547 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.550957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9053142-366e-4b29-93f5-c2413022cfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.551633 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9053142-366e-4b29-93f5-c2413022cfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.551911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9053142-366e-4b29-93f5-c2413022cfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.552282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9053142-366e-4b29-93f5-c2413022cfd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.553086 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9053142-366e-4b29-93f5-c2413022cfd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.555331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9053142-366e-4b29-93f5-c2413022cfd3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.568948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1387/2780] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:40:29.573711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.590303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9cea0-0c38-4550-9f7b-84ce341612c2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.596979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.597760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.598047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.598327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] PATH_INFO: `/v3/projects/f7b68fe7fb1e4fbbb2ad95ed00c426ab/users/df796027d20b4432bc3029fa848a06d4/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.615001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b68fe7fb1e4fbbb2ad95ed00c426ab, user_id=df796027d20b4432bc3029fa848a06d4, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.617718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.637455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1394/2781] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:40:29 2022] PUT /identity/v3/projects/f7b68fe7fb1e4fbbb2ad95ed00c426ab/users/df796027d20b4432bc3029fa848a06d4/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:40:29.648808 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676d598-3a45-4bbd-9f00-b7ebc5be7bfc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.649097 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676d598-3a45-4bbd-9f00-b7ebc5be7bfc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.649366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676d598-3a45-4bbd-9f00-b7ebc5be7bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.681468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3676d598-3a45-4bbd-9f00-b7ebc5be7bfc None None] MFA Rules not processed for user `df796027d20b4432bc3029fa848a06d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:40:29.683457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3676d598-3a45-4bbd-9f00-b7ebc5be7bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:29.732090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:29.762991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1388/2782] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:40:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:40:29.781116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:40:29.786156 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:40:29.836789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:29.883973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f27aed-f1ea-4cae-aa78-2de94529fdc1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:40:29.888072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b451-5a0e-442d-a39d-ef04d5776128 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05a016553950460dbb736ced2cae4f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:40:29.888872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b451-5a0e-442d-a39d-ef04d5776128 service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:40:29.888970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b451-5a0e-442d-a39d-ef04d5776128 service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:40:29.889208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b451-5a0e-442d-a39d-ef04d5776128 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:40:29.898650 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b451-5a0e-442d-a39d-ef04d5776128 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:40:29.900097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b451-5a0e-442d-a39d-ef04d5776128 service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:40:29.910888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1395/2783] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:40:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 05 17:40:39.758379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=8 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:40:39.906576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=19 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:04.413883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676d598-3a45-4bbd-9f00-b7ebc5be7bfc None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:04.480291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676d598-3a45-4bbd-9f00-b7ebc5be7bfc None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:04.485309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:04.485939 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:04.486216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:04.486484 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] PATH_INFO: `/v3/users/df796027d20b4432bc3029fa848a06d4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:04.487588 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=df796027d20b4432bc3029fa848a06d4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:04.489809 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:04.543241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df796027d20b4432bc3029fa848a06d4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:04.543903 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df796027d20b4432bc3029fa848a06d4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:04.553929 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df796027d20b4432bc3029fa848a06d4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:04.566892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df796027d20b4432bc3029fa848a06d4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:04.567705 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df796027d20b4432bc3029fa848a06d4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:04.580623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1389/2784] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:41:04 2022] DELETE /identity/v3/users/df796027d20b4432bc3029fa848a06d4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:41:04.585800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:04.586646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:04.586646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:04.606228 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:04.607430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:04.669421 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1396/2785] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:04.682634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:04.684206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:04.747660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccaa66d-7db2-493e-a14d-33ba357153dd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:04.752591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6b6a2-9036-4bc3-8085-0dace7e3cc5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:04.753208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6b6a2-9036-4bc3-8085-0dace7e3cc5c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:04.753482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6b6a2-9036-4bc3-8085-0dace7e3cc5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:04.753757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6b6a2-9036-4bc3-8085-0dace7e3cc5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:04.764530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6b6a2-9036-4bc3-8085-0dace7e3cc5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:04.765976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6b6a2-9036-4bc3-8085-0dace7e3cc5c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:04.777832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1390/2786] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:05.064092 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.065663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:05.078559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eaf40-6281-4c30-93d0-1fdb35774b1c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.084197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.084861 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.085145 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.085431 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] PATH_INFO: `/v3/projects/f7b68fe7fb1e4fbbb2ad95ed00c426ab` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.092903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b68fe7fb1e4fbbb2ad95ed00c426ab)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.094836 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.147954 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b68fe7fb1e4fbbb2ad95ed00c426ab', 'request_id': 'req-4f1f33ef-30a3-4786-9881-3dac5bded8a4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:05.154425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b68fe7fb1e4fbbb2ad95ed00c426ab', 'request_id': 'req-4f1f33ef-30a3-4786-9881-3dac5bded8a4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:05.161109 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b68fe7fb1e4fbbb2ad95ed00c426ab', 'request_id': 'req-4f1f33ef-30a3-4786-9881-3dac5bded8a4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:05.175999 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] The token cache is being invalidate because project f7b68fe7fb1e4fbbb2ad95ed00c426ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:41:05.176216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:05.176911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1f33ef-30a3-4786-9881-3dac5bded8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:05.178534 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1397/2787] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:41:05 2022] DELETE /identity/v3/projects/f7b68fe7fb1e4fbbb2ad95ed00c426ab => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:41:05.183652 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347882fc-b25f-4853-a1f1-8124c233646a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.183936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347882fc-b25f-4853-a1f1-8124c233646a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.184210 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347882fc-b25f-4853-a1f1-8124c233646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.204731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-347882fc-b25f-4853-a1f1-8124c233646a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:05.205927 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-347882fc-b25f-4853-a1f1-8124c233646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:05.224607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:05.269279 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1391/2788] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:05.277355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d03ac-34e0-444e-8e22-811db0e6ddd5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.277637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d03ac-34e0-444e-8e22-811db0e6ddd5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.277901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d03ac-34e0-444e-8e22-811db0e6ddd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.298814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ec4d03ac-34e0-444e-8e22-811db0e6ddd5 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:05.300018 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ec4d03ac-34e0-444e-8e22-811db0e6ddd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:05.307840 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1398/2789] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:05.312637 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347882fc-b25f-4853-a1f1-8124c233646a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.324963 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347882fc-b25f-4853-a1f1-8124c233646a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.330576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.331208 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.331470 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.331744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.332469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:41:05.332867 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.334500 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.346960 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1392/2790] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:41:05.351609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d03ac-34e0-444e-8e22-811db0e6ddd5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.361325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d03ac-34e0-444e-8e22-811db0e6ddd5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.366947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.367563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.367837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.368142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.368806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.370737 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.405957 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1399/2791] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:41:05 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:41:05.410654 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.421526 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb73752-5038-4ed0-8dde-f341b668a385 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.484300 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.484834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.485108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.485377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.486002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.487910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.511951 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1393/2792] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:41:05 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:41:05.516389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.527460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd0708-e060-40f2-b766-e8d0e8718a7b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.532529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.533126 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.533397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.533660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.534326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.536197 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.546408 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1400/2793] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:41:05.550410 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.560838 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9131c9c-e99c-478a-beca-79dc171cb13d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.565456 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef657e24-e132-4189-94d8-2f97455d9c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.566047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657e24-e132-4189-94d8-2f97455d9c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.566323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657e24-e132-4189-94d8-2f97455d9c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.566592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef657e24-e132-4189-94d8-2f97455d9c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.567230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef657e24-e132-4189-94d8-2f97455d9c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.568849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef657e24-e132-4189-94d8-2f97455d9c8c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.579713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1394/2794] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:41:05.584247 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.595129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed0516-5e31-41f1-a43b-7303a8f4d701 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.601311 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.601955 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.602256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.602502 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] PATH_INFO: `/v3/projects/448baffb86284acda27f4470b2c85c07/users/e0d73370a2e54b92b3094522335466d9/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.618530 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448baffb86284acda27f4470b2c85c07, user_id=e0d73370a2e54b92b3094522335466d9, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.621271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.641832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1401/2795] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:41:05 2022] PUT /identity/v3/projects/448baffb86284acda27f4470b2c85c07/users/e0d73370a2e54b92b3094522335466d9/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:41:05.711013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996bd7-cc7c-48c1-81c9-180d3d05da85 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.711687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996bd7-cc7c-48c1-81c9-180d3d05da85 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.712121 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5996bd7-cc7c-48c1-81c9-180d3d05da85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.750001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f5996bd7-cc7c-48c1-81c9-180d3d05da85 None None] MFA Rules not processed for user `e0d73370a2e54b92b3094522335466d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:05.751158 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f5996bd7-cc7c-48c1-81c9-180d3d05da85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:05.819374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1395/2796] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:05.832073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:05.834355 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:05.857296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:05.895414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba735abb-3ebd-461e-b362-20e6ae5e7410 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:05.899572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:05.900220 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:05.900517 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:05.900781 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:05.909446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:05.911147 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:05.920346 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1402/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:07.305199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:07.305754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:07.305754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:07.326010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] MFA Rules not processed for user `e0d73370a2e54b92b3094522335466d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:07.327309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:07.335080 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1396/2798] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:08.372774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:08.374236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:08.478702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9c890-05fe-41db-9689-03c0c8c682dd service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:08.484475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85a572-eee4-47f2-9dee-7bee2b8debf1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:08.485115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a572-eee4-47f2-9dee-7bee2b8debf1 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:08.485395 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a572-eee4-47f2-9dee-7bee2b8debf1 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:08.485666 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85a572-eee4-47f2-9dee-7bee2b8debf1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:08.487614 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a85a572-eee4-47f2-9dee-7bee2b8debf1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:08.500535 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85a572-eee4-47f2-9dee-7bee2b8debf1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:08.502120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85a572-eee4-47f2-9dee-7bee2b8debf1 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:08.513356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1403/2799] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:17.320477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:18.508460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=58 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:23.247370 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:23.249880 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:23.360387 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6dbdb-b23b-45fe-889f-efbf0b5d3a89 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:23.366102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:23.367029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:23.367359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:23.367630 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:23.369888 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:23.383687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:23.385424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:23.396564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1397/2800] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:24.873549 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4697db-135c-4e3e-90d0-2367ba3a8323 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:24.874073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4697db-135c-4e3e-90d0-2367ba3a8323 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:24.874073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4697db-135c-4e3e-90d0-2367ba3a8323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:24.894632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dc4697db-135c-4e3e-90d0-2367ba3a8323 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:24.895936 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dc4697db-135c-4e3e-90d0-2367ba3a8323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:24.957325 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1404/2801] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:41:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:25.015856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:25.017592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:25.092627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a4cb7-a861-47f0-a789-8f9d95b9efd7 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:25.097884 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:25.098488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:25.098763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:25.099027 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:25.108115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:25.109731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:25.118072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1398/2802] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:41:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:41:25.677152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc269-fc47-4499-a1a3-e652ffbcda5a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:25.677442 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc269-fc47-4499-a1a3-e652ffbcda5a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:25.677718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bc269-fc47-4499-a1a3-e652ffbcda5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:25.697787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-319bc269-fc47-4499-a1a3-e652ffbcda5a None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:25.698987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-319bc269-fc47-4499-a1a3-e652ffbcda5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:25.706658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1405/2803] 10.222.0.23 () {58 vars in 967 bytes} [Mon Sep 5 17:41:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:25.719511 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:25.731242 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc56674-fc05-41a7-908f-e9a91e2bed1e service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:25.737092 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:25.737687 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:25.737965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:25.738235 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:25.747564 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:25.749183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:25.757892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1399/2804] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:41:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:41:26.013742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fbfd-7bbf-43bc-81cb-59067546912e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:26.014217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fbfd-7bbf-43bc-81cb-59067546912e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:26.014320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259fbfd-7bbf-43bc-81cb-59067546912e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:26.034455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e259fbfd-7bbf-43bc-81cb-59067546912e None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:26.035657 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e259fbfd-7bbf-43bc-81cb-59067546912e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:26.044823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1406/2805] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Sep 5 17:41:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:26.059781 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:26.071379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e2376-ab4c-47bb-a4d0-260bbcf55b67 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:26.080257 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:26.081076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:26.081401 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:26.081724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:26.092837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:26.094409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:26.103166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1400/2806] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:41:26.431031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75735c-618a-47a0-896b-3de34430c5a8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:26.431317 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75735c-618a-47a0-896b-3de34430c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:26.431583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75735c-618a-47a0-896b-3de34430c5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:26.452863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1c75735c-618a-47a0-896b-3de34430c5a8 None None] MFA Rules not processed for user `2f2310a3da094b959270d797bd2f1553`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:26.454084 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1c75735c-618a-47a0-896b-3de34430c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:26.462583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1407/2807] 10.222.0.23 () {58 vars in 967 bytes} [Mon Sep 5 17:41:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:26.475261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:26.486852 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e993-1413-49c9-aade-60d2046b5851 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:26.492778 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b666b-fb5f-4a92-9d1f-e97f7592148b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2756036297064cc19d5fd5d1861703c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:26.493489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b666b-fb5f-4a92-9d1f-e97f7592148b service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:26.493772 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b666b-fb5f-4a92-9d1f-e97f7592148b service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:26.494243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b666b-fb5f-4a92-9d1f-e97f7592148b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:26.503385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b666b-fb5f-4a92-9d1f-e97f7592148b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:26.505021 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b666b-fb5f-4a92-9d1f-e97f7592148b service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:26.514034 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1401/2808] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Sep 5 17:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 05 17:41:28.703040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75735c-618a-47a0-896b-3de34430c5a8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:28.780754 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75735c-618a-47a0-896b-3de34430c5a8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:28.785958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:28.786598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:28.786879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:28.787146 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] PATH_INFO: `/v3/users/e0d73370a2e54b92b3094522335466d9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:28.788425 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d73370a2e54b92b3094522335466d9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:28.790691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:28.849659 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d73370a2e54b92b3094522335466d9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:28.860880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d73370a2e54b92b3094522335466d9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:28.875931 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d73370a2e54b92b3094522335466d9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:28.876819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d73370a2e54b92b3094522335466d9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:28.887233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d73370a2e54b92b3094522335466d9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:28.895713 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1408/2809] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:41:28 2022] DELETE /identity/v3/users/e0d73370a2e54b92b3094522335466d9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:41:28.903129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:28.903928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:28.905241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:28.931905 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:28.933217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:28.978129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:29.023703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1402/2810] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:29.029850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.031559 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:29.039600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:29.117060 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce47261-f933-430e-b0e1-202c8249be8f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.122791 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd09fa5-5809-4b92-8f5c-c7d5cd8f4c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.123423 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09fa5-5809-4b92-8f5c-c7d5cd8f4c2d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.123706 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09fa5-5809-4b92-8f5c-c7d5cd8f4c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.123976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd09fa5-5809-4b92-8f5c-c7d5cd8f4c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.135854 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd09fa5-5809-4b92-8f5c-c7d5cd8f4c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.137372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd09fa5-5809-4b92-8f5c-c7d5cd8f4c2d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.147965 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1409/2811] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:29.346096 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.347593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:29.363524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b7931-5c7a-4be4-ae66-98e464fef149 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.371274 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.371897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.372391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.372669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] PATH_INFO: `/v3/projects/448baffb86284acda27f4470b2c85c07` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.373968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448baffb86284acda27f4470b2c85c07)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.384038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.435934 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448baffb86284acda27f4470b2c85c07', 'request_id': 'req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:29.441701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448baffb86284acda27f4470b2c85c07', 'request_id': 'req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:29.448431 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448baffb86284acda27f4470b2c85c07', 'request_id': 'req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:29.464493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] The token cache is being invalidate because project 448baffb86284acda27f4470b2c85c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:41:29.464786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:29.465436 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-da92c71c-f86b-4c54-9ee6-ebca9270f8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:29.466987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1403/2812] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:41:29 2022] DELETE /identity/v3/projects/448baffb86284acda27f4470b2c85c07 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:41:29.472752 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50797-43c5-45e8-bc4f-bd442ac665ee None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.473128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50797-43c5-45e8-bc4f-bd442ac665ee None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.473512 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50797-43c5-45e8-bc4f-bd442ac665ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.497134 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3fb50797-43c5-45e8-bc4f-bd442ac665ee None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:29.498878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3fb50797-43c5-45e8-bc4f-bd442ac665ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:29.564525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1410/2813] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:29.576194 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998082e-3033-4afa-a9c0-9a33d2f56574 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.576512 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998082e-3033-4afa-a9c0-9a33d2f56574 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.576785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998082e-3033-4afa-a9c0-9a33d2f56574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.598231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1998082e-3033-4afa-a9c0-9a33d2f56574 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:29.599428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1998082e-3033-4afa-a9c0-9a33d2f56574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:29.607967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1404/2814] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:29.612691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50797-43c5-45e8-bc4f-bd442ac665ee None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.623353 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50797-43c5-45e8-bc4f-bd442ac665ee None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.630258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.631080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.631429 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.631790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.632480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:41:29.632871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.635222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.647710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1411/2815] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:41:29.652602 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998082e-3033-4afa-a9c0-9a33d2f56574 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.662693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998082e-3033-4afa-a9c0-9a33d2f56574 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.668697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.669311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.669586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.669849 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.670468 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.672433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.704048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1405/2816] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:41:29 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:41:29.712958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.723097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f57b-10da-48a7-b602-28decdec8792 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.781515 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.782124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.782413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.782658 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.783306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.785274 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.809699 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1412/2817] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:41:29 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:41:29.813924 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.824510 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01207634-216b-4ea8-bc36-ae0093d9fbad admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.830113 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.830718 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.830992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.831255 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.831891 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.833553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.844682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1406/2818] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:41:29.848888 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.860184 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813c835-79c2-48fd-ac21-e8f83bb3846b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.865364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9172f0-b318-4032-9cfd-3a6efc313686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.865987 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9172f0-b318-4032-9cfd-3a6efc313686 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.866265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9172f0-b318-4032-9cfd-3a6efc313686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.866532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9172f0-b318-4032-9cfd-3a6efc313686 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.867150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9172f0-b318-4032-9cfd-3a6efc313686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.868930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9172f0-b318-4032-9cfd-3a6efc313686 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.880673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1413/2819] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:41:29.884923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:29.895346 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63d8b6-a718-4b60-af8b-117c2363cc4f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:29.901310 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:29.901964 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.902236 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.902502 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] PATH_INFO: `/v3/projects/252f2f027e06411eb72bcd2b30fbc903/users/280cb0ac6d35458a9463bd04167627db/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.918213 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=252f2f027e06411eb72bcd2b30fbc903, user_id=280cb0ac6d35458a9463bd04167627db, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:29.920936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:29.939323 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1407/2820] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:41:29 2022] PUT /identity/v3/projects/252f2f027e06411eb72bcd2b30fbc903/users/280cb0ac6d35458a9463bd04167627db/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:41:29.954120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94d30-c3e6-46ce-8421-67082c85d8d5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:29.954398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94d30-c3e6-46ce-8421-67082c85d8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:29.954665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94d30-c3e6-46ce-8421-67082c85d8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:29.980711 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:29.984379 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5ff94d30-c3e6-46ce-8421-67082c85d8d5 None None] MFA Rules not processed for user `280cb0ac6d35458a9463bd04167627db`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:29.986129 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5ff94d30-c3e6-46ce-8421-67082c85d8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:30.052388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1414/2821] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:30.065051 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:30.066983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:30.124576 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d967b7-bf08-4f7e-8dbc-e9f75a3c90c5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:30.128858 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d48f62-3d71-4063-9792-b540d5139934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:30.129459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d48f62-3d71-4063-9792-b540d5139934 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:30.129738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d48f62-3d71-4063-9792-b540d5139934 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:30.130005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d48f62-3d71-4063-9792-b540d5139934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:30.139314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d48f62-3d71-4063-9792-b540d5139934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:30.140686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d48f62-3d71-4063-9792-b540d5139934 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:30.144439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:30.151921 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1408/2822] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:40.049196 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=26 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:40.156347 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:52.835444 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff94d30-c3e6-46ce-8421-67082c85d8d5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:52.917558 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff94d30-c3e6-46ce-8421-67082c85d8d5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:52.922563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:52.923206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:52.923480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:52.923871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] PATH_INFO: `/v3/users/280cb0ac6d35458a9463bd04167627db` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:52.925203 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=280cb0ac6d35458a9463bd04167627db)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:52.927772 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:52.979542 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280cb0ac6d35458a9463bd04167627db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:52.988108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280cb0ac6d35458a9463bd04167627db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.006522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280cb0ac6d35458a9463bd04167627db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.007252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280cb0ac6d35458a9463bd04167627db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.015132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280cb0ac6d35458a9463bd04167627db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.021839 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1415/2823] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:41:52 2022] DELETE /identity/v3/users/280cb0ac6d35458a9463bd04167627db => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:41:53.027171 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.027704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.027704 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.048389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:53.049612 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:53.113140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1409/2824] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:53.123730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:53.125269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:53.187663 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8903918-3aea-4fb8-ad96-4ed65c687a0f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:53.191437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f42a2-3c57-4d5e-8db1-aebd7bbe6843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:53.192051 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f42a2-3c57-4d5e-8db1-aebd7bbe6843 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.192354 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f42a2-3c57-4d5e-8db1-aebd7bbe6843 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.192623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f42a2-3c57-4d5e-8db1-aebd7bbe6843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.202510 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f42a2-3c57-4d5e-8db1-aebd7bbe6843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:53.203953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f42a2-3c57-4d5e-8db1-aebd7bbe6843 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:53.213992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1416/2825] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:53.499261 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:53.501859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:53.515592 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3efa3-806a-4ad9-b8e1-48b2e9a01d27 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:53.520449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:53.521166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.521539 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.521706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] PATH_INFO: `/v3/projects/252f2f027e06411eb72bcd2b30fbc903` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.522861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252f2f027e06411eb72bcd2b30fbc903)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:53.524937 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:53.575837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252f2f027e06411eb72bcd2b30fbc903', 'request_id': 'req-5864f042-7c8c-464f-a385-4bd31e1f62e4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.581998 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252f2f027e06411eb72bcd2b30fbc903', 'request_id': 'req-5864f042-7c8c-464f-a385-4bd31e1f62e4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.589053 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252f2f027e06411eb72bcd2b30fbc903', 'request_id': 'req-5864f042-7c8c-464f-a385-4bd31e1f62e4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.603902 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] The token cache is being invalidate because project 252f2f027e06411eb72bcd2b30fbc903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:41:53.604223 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.604855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5864f042-7c8c-464f-a385-4bd31e1f62e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:41:53.606429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1410/2826] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:41:53 2022] DELETE /identity/v3/projects/252f2f027e06411eb72bcd2b30fbc903 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:41:53.611715 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd884c0-b574-457c-9c16-67bf5865db67 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.611998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd884c0-b574-457c-9c16-67bf5865db67 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.612308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd884c0-b574-457c-9c16-67bf5865db67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.635202 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dfd884c0-b574-457c-9c16-67bf5865db67 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:53.636356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dfd884c0-b574-457c-9c16-67bf5865db67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:53.654009 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:53.697902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1417/2827] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:53.707185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4702e-b6f2-4f16-9042-ccbef2c013a3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.707476 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4702e-b6f2-4f16-9042-ccbef2c013a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.707746 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4702e-b6f2-4f16-9042-ccbef2c013a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.728682 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-87a4702e-b6f2-4f16-9042-ccbef2c013a3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:53.729877 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-87a4702e-b6f2-4f16-9042-ccbef2c013a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:53.738028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1411/2828] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:53.742362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd884c0-b574-457c-9c16-67bf5865db67 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:53.752296 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd884c0-b574-457c-9c16-67bf5865db67 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:53.761114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:53.761725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.762001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.762271 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.762911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:41:53.763214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:53.764846 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:53.776860 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1418/2829] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:41:53.781719 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4702e-b6f2-4f16-9042-ccbef2c013a3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:53.793068 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4702e-b6f2-4f16-9042-ccbef2c013a3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:53.798599 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:53.799207 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.799486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.799758 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.800439 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:53.802881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:53.837424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1412/2830] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:41:53 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:41:53.841833 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:53.853329 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a4ea1-5d8a-4863-8942-c34035e76b5c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:53.910046 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:53.910574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.910848 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.911118 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.911766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:53.913849 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:53.936670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1419/2831] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:41:53 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:41:53.940919 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:53.951246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645ddf-f44e-40fe-a3a5-545136833519 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:53.956036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:53.956671 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.956968 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.957234 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.957868 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:53.959749 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:53.970385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1413/2832] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:41:53.974332 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:53.984842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058736-0a1b-4184-9579-6faa35574036 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:53.989993 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b53c9-9e56-4c55-9568-b3291ac8201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:53.990691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b53c9-9e56-4c55-9568-b3291ac8201a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:53.990976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b53c9-9e56-4c55-9568-b3291ac8201a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:53.991281 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b53c9-9e56-4c55-9568-b3291ac8201a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:53.991920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b53c9-9e56-4c55-9568-b3291ac8201a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:53.993623 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b53c9-9e56-4c55-9568-b3291ac8201a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:54.004565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1420/2833] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:41:54.009332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:54.024282 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea52d4-4ca8-476a-90cc-5b37dcced829 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:54.029815 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:54.030422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:54.030714 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:54.030993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] PATH_INFO: `/v3/projects/668b8711b6fd422b9e58f852c1801569/users/f0a8514ae867426c80200fbc90435c6d/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:54.047173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=668b8711b6fd422b9e58f852c1801569, user_id=f0a8514ae867426c80200fbc90435c6d, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:54.049892 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:54.066076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1414/2834] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:41:54 2022] PUT /identity/v3/projects/668b8711b6fd422b9e58f852c1801569/users/f0a8514ae867426c80200fbc90435c6d/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:41:54.071574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4200e-0927-49c2-9641-978e5ca8ce5b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:54.071868 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4200e-0927-49c2-9641-978e5ca8ce5b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:54.072142 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4200e-0927-49c2-9641-978e5ca8ce5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:54.092726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-57b4200e-0927-49c2-9641-978e5ca8ce5b None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:54.093924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-57b4200e-0927-49c2-9641-978e5ca8ce5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:54.157740 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1421/2835] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:54.168173 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:54.170358 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:54.193936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:54.237553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c6a74-07e0-4229-97df-9908a54da1c2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:54.242160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:54.242787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:54.243037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:54.243289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:54.253154 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:54.254624 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:54.265351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1415/2836] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:41:58.534097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28c21-6471-46cd-971e-71ead97827d0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:58.534943 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28c21-6471-46cd-971e-71ead97827d0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:58.535222 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28c21-6471-46cd-971e-71ead97827d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:58.565700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1fa28c21-6471-46cd-971e-71ead97827d0 None None] MFA Rules not processed for user `f0a8514ae867426c80200fbc90435c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:41:58.567056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1fa28c21-6471-46cd-971e-71ead97827d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:41:58.585113 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:41:58.641826 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1422/2837] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:41:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:41:58.647348 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:58.658388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cde617-2a35-45bd-93fe-058619e1e7f3 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:58.664054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:58.664811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:58.665157 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:58.665494 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:58.666218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:58.668244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:58.699455 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1416/2838] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:41:58 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:41:58.703640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa28c21-6471-46cd-971e-71ead97827d0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:58.715536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa28c21-6471-46cd-971e-71ead97827d0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:58.773687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:58.774320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:58.774604 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:58.774878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:58.775538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:58.777564 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:58.803206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1423/2839] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:41:58 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:41:58.809356 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:58.821997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada29a6-0918-48ed-bd07-900bf08c4861 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:58.826954 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:58.827674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:58.828014 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:58.828374 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:58.829087 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:58.830763 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:58.848409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1417/2840] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:41:58.849941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:58.860465 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cb31a-5feb-4746-bc06-721a930c0d64 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:58.865582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:58.866285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:58.866588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:58.866858 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] PATH_INFO: `/v3/projects/2064744cd17045e9ae2a74670c9f70fb/users/0f69ea3b15e944e1832d2230cd1b8bb8/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:58.882553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2064744cd17045e9ae2a74670c9f70fb, user_id=0f69ea3b15e944e1832d2230cd1b8bb8, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:58.885285 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:58.901609 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1424/2841] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:41:58 2022] PUT /identity/v3/projects/2064744cd17045e9ae2a74670c9f70fb/users/0f69ea3b15e944e1832d2230cd1b8bb8/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:41:58.905857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:58.916881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61f679-dbc6-405f-ab3f-c2c9e4c9df36 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:58.983469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f441b-f2f7-4477-96ba-fb7dbae1af2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:58.984103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f441b-f2f7-4477-96ba-fb7dbae1af2e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:58.984424 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f441b-f2f7-4477-96ba-fb7dbae1af2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:58.984696 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f441b-f2f7-4477-96ba-fb7dbae1af2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:58.985342 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f441b-f2f7-4477-96ba-fb7dbae1af2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:58.987026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f441b-f2f7-4477-96ba-fb7dbae1af2e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:58.998008 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1418/2842] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:41:59.002433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:41:59.013050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09857919-44ac-4a7d-b0c0-35f2f34704a5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:41:59.018306 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:41:59.018979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:41:59.019262 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:41:59.019539 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] PATH_INFO: `/v3/projects/2064744cd17045e9ae2a74670c9f70fb/users/0f69ea3b15e944e1832d2230cd1b8bb8/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:41:59.021742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2064744cd17045e9ae2a74670c9f70fb, user_id=0f69ea3b15e944e1832d2230cd1b8bb8, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:41:59.024440 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:41:59.034917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1425/2843] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:41:59 2022] PUT /identity/v3/projects/2064744cd17045e9ae2a74670c9f70fb/users/0f69ea3b15e944e1832d2230cd1b8bb8/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:42:03.565557 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73a81-515f-468c-843d-93455435b2db None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:42:03.566057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73a81-515f-468c-843d-93455435b2db None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:42:03.566163 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b73a81-515f-468c-843d-93455435b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:42:03.594722 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-85b73a81-515f-468c-843d-93455435b2db None None] MFA Rules not processed for user `0f69ea3b15e944e1832d2230cd1b8bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:42:03.595962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-85b73a81-515f-468c-843d-93455435b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:42:03.678974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1419/2844] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:42:03.702430 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:42:03.706016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:42:03.794392 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cae49-a412-41f0-90a8-d74c6da4ad25 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:42:03.799850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:42:03.800499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:42:03.800776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:42:03.801053 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:42:03.807144 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:42:03.863028 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:42:03.864787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:42:03.875625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1426/2845] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:42:04.962460 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:42:04.962992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:42:04.964382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:42:04.987046 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] MFA Rules not processed for user `f0a8514ae867426c80200fbc90435c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:42:04.988361 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:42:04.996419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1420/2846] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:42:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:42:13.870224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=24 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:42:14.982296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:42:29.191265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:42:29.281746 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ff0ed-d5d5-47a1-b835-e480b71759c9 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:42:29.285880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:42:29.286477 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:42:29.286729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:42:29.287024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:42:29.288898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:42:29.311672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:42:29.313159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:42:29.321398 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1427/2847] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:42:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:42:29.797509 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:42:29.798986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:42:29.910928 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ecf550-ceed-47f1-bf4e-1f75516a7542 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:42:29.916054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:42:29.916716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:42:29.916996 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:42:29.917270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:42:29.918986 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:42:29.931628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:42:29.933276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:42:29.944289 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1421/2848] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:42:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:42:39.317851 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=43 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:42:39.939701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=47 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:42:44.789301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:42:44.825414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06314251-5989-4a5a-9bba-819544f2b94d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:42:44.833593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:42:44.833835 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:42:44.834163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:42:44.834437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:42:44.836774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:42:44.850370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:42:44.852017 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:42:44.863305 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1428/2849] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:42:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:42:47.021598 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:42:47.031769 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7faebe-7b3e-4ed3-929f-d067ba641470 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:42:47.036317 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:42:47.036918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:42:47.037198 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:42:47.037466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:42:47.047232 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:42:47.048708 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:42:47.058737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1422/2850] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:42:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:42:54.858730 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=13 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:42:57.054309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:43:15.503436 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:43:15.514453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76571b-6786-4384-abcf-1097c0ea28a6 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:43:15.519054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:43:15.519689 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:43:15.519964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:43:15.520261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:43:15.522524 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:43:15.533789 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:43:15.535323 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:43:15.543923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1429/2851] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:43:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:43:25.540842 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:15.751070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:15.760515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8ab9-86a5-49c3-9f64-f730b571e8ce service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:15.764485 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:15.765098 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:15.765375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:15.765646 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:15.767433 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:15.840915 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:15.843133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:15.853679 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1423/2852] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:19.356275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:19.370625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24632d4-9620-415b-9edb-b450157b3219 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:19.376380 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:19.377168 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:19.377447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:19.377799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:19.379588 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:19.392364 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:19.394024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:19.407404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1430/2853] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:19.412449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:19.413997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:19.427184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de1ead-8e79-4032-af2c-e71f7200787f service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:19.432699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:19.433322 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:19.433593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:19.433859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] PATH_INFO: `/v3/users/c53ac49fbc054c01a5a1ef41e04bf253` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:19.435003 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53ac49fbc054c01a5a1ef41e04bf253)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:19.436938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:19.486661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53ac49fbc054c01a5a1ef41e04bf253'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.487243 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53ac49fbc054c01a5a1ef41e04bf253'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.496009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53ac49fbc054c01a5a1ef41e04bf253'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.509620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53ac49fbc054c01a5a1ef41e04bf253'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.510334 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9cd8df5c-1334-477b-a9f1-8dc316f64645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53ac49fbc054c01a5a1ef41e04bf253'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.524489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1424/2854] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:19 2022] DELETE /identity/v3/users/c53ac49fbc054c01a5a1ef41e04bf253 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:19.708910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:19.729170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467ef883-c8d2-4ce8-a7f3-d76644d618b7 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:19.791514 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:19.792249 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:19.792474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:19.792742 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] PATH_INFO: `/v3/projects/e933bc308b31482a9db533a7e89aeb4b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:19.803774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e933bc308b31482a9db533a7e89aeb4b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:19.805795 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:19.854255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e933bc308b31482a9db533a7e89aeb4b', 'request_id': 'req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.860927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e933bc308b31482a9db533a7e89aeb4b', 'request_id': 'req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.867086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e933bc308b31482a9db533a7e89aeb4b', 'request_id': 'req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.880997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] The token cache is being invalidate because project e933bc308b31482a9db533a7e89aeb4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:19.881341 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.881994 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-541d36dd-55d7-4e55-9c24-fb7dc9ee742b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:19.883633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1431/2855] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:19 2022] DELETE /identity/v3/projects/e933bc308b31482a9db533a7e89aeb4b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:19.889469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6153e6e-74c2-4551-ac2a-83f3797645b4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:19.889783 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6153e6e-74c2-4551-ac2a-83f3797645b4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:19.890061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6153e6e-74c2-4551-ac2a-83f3797645b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:19.909967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c6153e6e-74c2-4551-ac2a-83f3797645b4 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:19.911180 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c6153e6e-74c2-4551-ac2a-83f3797645b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:19.977031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1425/2856] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:19.984801 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa742e3-3ac3-421a-9321-7c11c08b86ad None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:19.985069 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa742e3-3ac3-421a-9321-7c11c08b86ad None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:19.985370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa742e3-3ac3-421a-9321-7c11c08b86ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.008995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0fa742e3-3ac3-421a-9321-7c11c08b86ad None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:20.010214 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0fa742e3-3ac3-421a-9321-7c11c08b86ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:20.018594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1432/2857] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:20.023529 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6153e6e-74c2-4551-ac2a-83f3797645b4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.034081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6153e6e-74c2-4551-ac2a-83f3797645b4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.039404 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.040012 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.040332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.040600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.041250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:44:20.041549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.043137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.044811 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:20.057898 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1426/2858] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:44:20.061768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa742e3-3ac3-421a-9321-7c11c08b86ad None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.071428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa742e3-3ac3-421a-9321-7c11c08b86ad None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.076213 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.076971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.077252 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.077521 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.078163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.080063 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.111760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1433/2859] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:20 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:20.116193 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.126603 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647fe09-e522-424d-aac8-24404062d560 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.188807 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.189415 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.189691 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.189962 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.190583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.192524 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.216813 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1427/2860] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:20 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:20.221075 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.232050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f9488-c3d5-4ed3-aea5-993eea2e568a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.237373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.237908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.238190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.238464 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.239243 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.241040 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.251504 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1434/2861] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:20.255706 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.265757 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966bae6-abfb-4d76-adb3-f8950c98fd05 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.270699 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52ce37-38ca-4db4-9e5f-ba51f3eafb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.271298 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52ce37-38ca-4db4-9e5f-ba51f3eafb6a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.271582 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52ce37-38ca-4db4-9e5f-ba51f3eafb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.271894 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52ce37-38ca-4db4-9e5f-ba51f3eafb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.272581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52ce37-38ca-4db4-9e5f-ba51f3eafb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.274174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52ce37-38ca-4db4-9e5f-ba51f3eafb6a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.284864 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1428/2862] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:20.291239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.305920 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5231b7-7baf-44eb-b58b-a6a7d46a261f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.311536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.312186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.312501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.312832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] PATH_INFO: `/v3/projects/29c9e354f00a4031afd32692840373b6/users/6111ebdac50645a98b4538a4f6bae1a3/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.329125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29c9e354f00a4031afd32692840373b6, user_id=6111ebdac50645a98b4538a4f6bae1a3, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.331797 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.347985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1435/2863] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:20 2022] PUT /identity/v3/projects/29c9e354f00a4031afd32692840373b6/users/6111ebdac50645a98b4538a4f6bae1a3/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:20.359427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.359710 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.359982 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.390145 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] MFA Rules not processed for user `6111ebdac50645a98b4538a4f6bae1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:20.391359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:20.471566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1429/2864] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:44:20.482921 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.484979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:20.491874 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:20.567728 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a0ad1-60f8-4f23-837f-f9adcd2dd52d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.575445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.576388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.576669 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.576991 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.589407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.592056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.603508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1436/2865] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:20.626367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:20.628378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:20.710674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589808e0-9b52-409e-812b-923fa4e3084f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:20.715286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:20.715896 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:20.716189 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:20.716477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:20.718465 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:20.731081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:20.734183 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:20.736102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:20.748482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1430/2866] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:21.081352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:21.091995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85203ab-927e-46bf-aa93-ab5afc7c4e17 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:21.096204 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2629cc-ef5c-4bf5-b7b7-58756adde1fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:21.096832 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2629cc-ef5c-4bf5-b7b7-58756adde1fa service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:21.097083 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2629cc-ef5c-4bf5-b7b7-58756adde1fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:21.097349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2629cc-ef5c-4bf5-b7b7-58756adde1fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:21.099337 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8c2629cc-ef5c-4bf5-b7b7-58756adde1fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:21.114793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2629cc-ef5c-4bf5-b7b7-58756adde1fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:21.116340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2629cc-ef5c-4bf5-b7b7-58756adde1fa service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:21.127692 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1437/2867] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:23.547241 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:23.612661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb992fa-3c2a-4a76-b0e7-4115c9bd9959 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:23.617974 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:23.618626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:23.618893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:23.619153 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] PATH_INFO: `/v3/users/6111ebdac50645a98b4538a4f6bae1a3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:23.620414 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] RBAC: Authorizing `identity:delete_user(user_id=6111ebdac50645a98b4538a4f6bae1a3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:23.622365 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:23.678785 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6111ebdac50645a98b4538a4f6bae1a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:23.679391 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6111ebdac50645a98b4538a4f6bae1a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:23.687695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6111ebdac50645a98b4538a4f6bae1a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:23.700724 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6111ebdac50645a98b4538a4f6bae1a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:23.701428 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6111ebdac50645a98b4538a4f6bae1a3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:23.714532 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1431/2868] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:23 2022] DELETE /identity/v3/users/6111ebdac50645a98b4538a4f6bae1a3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:23.720314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:23.721043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:23.721370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:23.748508 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:23.749743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:23.814552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1438/2869] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:23.824674 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:23.826195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:23.886767 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bca81-1d6a-4298-b4b9-ff7761bdecef admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:23.890949 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226c726-f545-4562-845f-d7a0fc9130d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:23.891552 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c726-f545-4562-845f-d7a0fc9130d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:23.891827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c726-f545-4562-845f-d7a0fc9130d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:23.892100 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226c726-f545-4562-845f-d7a0fc9130d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:23.903047 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226c726-f545-4562-845f-d7a0fc9130d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:23.904556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226c726-f545-4562-845f-d7a0fc9130d6 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:23.914600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1432/2870] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:24.200796 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.202254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:24.214866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdf79d-3941-4e48-a5a7-52cd0810c3fb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.219917 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.220603 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.220879 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.221149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] PATH_INFO: `/v3/projects/29c9e354f00a4031afd32692840373b6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.222393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29c9e354f00a4031afd32692840373b6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.224393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.276928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29c9e354f00a4031afd32692840373b6', 'request_id': 'req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:24.283385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29c9e354f00a4031afd32692840373b6', 'request_id': 'req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:24.289413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29c9e354f00a4031afd32692840373b6', 'request_id': 'req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:24.300044 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:24.304414 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] The token cache is being invalidate because project 29c9e354f00a4031afd32692840373b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:24.305023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:24.305910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b0e4f46f-9e4b-49df-a6d8-e4ed580a5026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:24.307655 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1439/2871] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:24 2022] DELETE /identity/v3/projects/29c9e354f00a4031afd32692840373b6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:44:24.312730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ace0-8918-42f8-a43a-81ae9ac84fbf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.313045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ace0-8918-42f8-a43a-81ae9ac84fbf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.313363 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ace0-8918-42f8-a43a-81ae9ac84fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.334508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b329ace0-8918-42f8-a43a-81ae9ac84fbf None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:24.336078 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b329ace0-8918-42f8-a43a-81ae9ac84fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:24.349246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:24.400911 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1433/2872] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:24.408855 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58281f1a-e12b-44dd-a4c2-93b4f9e77aee None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.409140 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58281f1a-e12b-44dd-a4c2-93b4f9e77aee None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.409412 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58281f1a-e12b-44dd-a4c2-93b4f9e77aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.430637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-58281f1a-e12b-44dd-a4c2-93b4f9e77aee None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:24.431853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-58281f1a-e12b-44dd-a4c2-93b4f9e77aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:24.441014 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1440/2873] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:24.445667 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ace0-8918-42f8-a43a-81ae9ac84fbf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.456544 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ace0-8918-42f8-a43a-81ae9ac84fbf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.463102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.463730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.463991 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.464299 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.464953 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:44:24.465283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.466936 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.479420 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1434/2874] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:44:24.483898 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58281f1a-e12b-44dd-a4c2-93b4f9e77aee None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.493694 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58281f1a-e12b-44dd-a4c2-93b4f9e77aee None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.499579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.500186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.500484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.500766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.501396 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.503362 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.534726 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1441/2875] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:24 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:24.541111 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.552832 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cada0-6700-47fe-9cf7-75c34e1f6fd7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.613872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.614482 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.614756 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.615024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.615653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.617616 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.641325 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1435/2876] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:24 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:24.645731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.656981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a4a81-d909-418b-8895-cd6a2bfa4240 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.662600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.663206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.663480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.663743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.664407 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.666004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.676185 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1442/2877] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:24.680611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.691159 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41b4bd-120d-438f-b28f-5e6dc8262175 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.697276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0c4be-5713-4ba1-bea3-394464f55bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.697836 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0c4be-5713-4ba1-bea3-394464f55bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.698138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0c4be-5713-4ba1-bea3-394464f55bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.698408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0c4be-5713-4ba1-bea3-394464f55bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.699048 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0c4be-5713-4ba1-bea3-394464f55bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.700692 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0c4be-5713-4ba1-bea3-394464f55bb3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.711565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1436/2878] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:24.715838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.726034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5244549-158f-4733-a6bb-3c003e04e0be admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.732190 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.733181 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.733563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.733911 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] PATH_INFO: `/v3/projects/24188593926b43ff9d3eec6805c8e359/users/aa6680141b204dd3a5449bde30bd682f/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.753424 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=24188593926b43ff9d3eec6805c8e359, user_id=aa6680141b204dd3a5449bde30bd682f, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.756139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.772769 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1443/2879] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:24 2022] PUT /identity/v3/projects/24188593926b43ff9d3eec6805c8e359/users/aa6680141b204dd3a5449bde30bd682f/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:24.783376 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa326eb-c0b0-45d1-83fc-050597c8d5da None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.783669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa326eb-c0b0-45d1-83fc-050597c8d5da None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.783946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa326eb-c0b0-45d1-83fc-050597c8d5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.814119 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dfa326eb-c0b0-45d1-83fc-050597c8d5da None None] MFA Rules not processed for user `aa6680141b204dd3a5449bde30bd682f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:24.815293 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dfa326eb-c0b0-45d1-83fc-050597c8d5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:24.882862 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1437/2880] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:24.887664 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:24.950859 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d235ed-cf04-4a1b-9343-a29f93e5f2ae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:24.955964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:24.956594 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:24.956844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:24.957135 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:24.957881 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:24.959787 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:24.990958 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1444/2881] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:24 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 05 17:44:24.997037 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa326eb-c0b0-45d1-83fc-050597c8d5da None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:25.013197 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa326eb-c0b0-45d1-83fc-050597c8d5da None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:25.028061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:25.083278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:25.083893 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:25.084195 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:25.084471 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:25.085452 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:25.087360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:25.110993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1438/2882] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:24 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:25.115193 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:25.125755 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a680516-4380-4611-b26f-bcad4d6eeb7a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:25.130470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:25.131073 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:25.131349 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:25.131619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:25.132282 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:25.133880 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:25.144579 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1445/2883] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:44:25.148660 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:25.159413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb50ee-e381-4817-b4fb-f81ef88ff0de admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:25.164419 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:25.165066 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:25.165339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:25.165607 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] PATH_INFO: `/v3/projects/5685a1babdfc42b1b1bb71074853737c/users/791a7cb1ae2b4b22aee3021fd26c543c/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:25.181108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5685a1babdfc42b1b1bb71074853737c, user_id=791a7cb1ae2b4b22aee3021fd26c543c, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:25.183801 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:25.198943 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1439/2884] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:25 2022] PUT /identity/v3/projects/5685a1babdfc42b1b1bb71074853737c/users/791a7cb1ae2b4b22aee3021fd26c543c/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:25.203108 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:25.213759 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffe999-c8fb-485c-b0b6-256e5e715846 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:25.273908 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed98186-58c2-428f-a63c-32d8d48a36a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:25.274492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98186-58c2-428f-a63c-32d8d48a36a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:25.274766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98186-58c2-428f-a63c-32d8d48a36a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:25.275031 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98186-58c2-428f-a63c-32d8d48a36a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:25.275834 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed98186-58c2-428f-a63c-32d8d48a36a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:25.277478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed98186-58c2-428f-a63c-32d8d48a36a6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:25.287893 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1446/2885] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:25.292116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:25.303230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb68d62-6b4e-45c3-98c7-3af15f2fe950 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:25.309253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:25.309923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:25.310199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:25.310474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] PATH_INFO: `/v3/projects/5685a1babdfc42b1b1bb71074853737c/users/791a7cb1ae2b4b22aee3021fd26c543c/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:25.312486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5685a1babdfc42b1b1bb71074853737c, user_id=791a7cb1ae2b4b22aee3021fd26c543c, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:25.315555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:25.326695 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1440/2886] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:25 2022] PUT /identity/v3/projects/5685a1babdfc42b1b1bb71074853737c/users/791a7cb1ae2b4b22aee3021fd26c543c/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:25.342899 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f6549-6b17-4c52-916e-9e27e96febd5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:25.343173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f6549-6b17-4c52-916e-9e27e96febd5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:25.343447 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f6549-6b17-4c52-916e-9e27e96febd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:25.358120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:25.375665 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-416f6549-6b17-4c52-916e-9e27e96febd5 None None] MFA Rules not processed for user `791a7cb1ae2b4b22aee3021fd26c543c`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:25.376913 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-416f6549-6b17-4c52-916e-9e27e96febd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:25.449788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1447/2887] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:25.461545 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:25.464406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:25.529950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43017f77-9c6d-436e-a853-de1c47635825 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:25.534409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc7ba9-1aab-4e60-a6f5-772f018e085f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:25.535013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7ba9-1aab-4e60-a6f5-772f018e085f service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:25.535290 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7ba9-1aab-4e60-a6f5-772f018e085f service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:25.535548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc7ba9-1aab-4e60-a6f5-772f018e085f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:25.545085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc7ba9-1aab-4e60-a6f5-772f018e085f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:25.546538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc7ba9-1aab-4e60-a6f5-772f018e085f service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:25.555977 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1441/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:27.005570 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f6549-6b17-4c52-916e-9e27e96febd5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:27.070779 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f6549-6b17-4c52-916e-9e27e96febd5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:27.075814 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:27.076488 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:27.076771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:27.077029 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] PATH_INFO: `/v3/users/aa6680141b204dd3a5449bde30bd682f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:27.078625 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6680141b204dd3a5449bde30bd682f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:27.080619 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:27.131455 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6680141b204dd3a5449bde30bd682f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.141469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6680141b204dd3a5449bde30bd682f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.155553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6680141b204dd3a5449bde30bd682f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.156345 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6680141b204dd3a5449bde30bd682f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.163503 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6680141b204dd3a5449bde30bd682f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.170052 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1448/2889] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:27 2022] DELETE /identity/v3/users/aa6680141b204dd3a5449bde30bd682f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:27.174855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:27.175320 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:27.175411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:27.194648 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:27.195859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:27.257750 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1442/2890] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:27.268087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:27.269660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:27.284719 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:27.334114 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43767d8-40ee-42ff-8fdb-4756d0185615 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:27.339056 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:27.339678 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:27.339947 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:27.340241 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:27.350094 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:27.351644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:27.362078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1449/2891] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:27.648622 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:27.650339 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:27.663466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88db52-9b88-4e5e-98f4-44e52be4156a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:27.673965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:27.674629 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:27.674913 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:27.675169 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] PATH_INFO: `/v3/projects/24188593926b43ff9d3eec6805c8e359` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:27.676350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24188593926b43ff9d3eec6805c8e359)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:27.678295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:27.681126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:27.728408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24188593926b43ff9d3eec6805c8e359', 'request_id': 'req-4c131ee0-64e8-44df-9b8a-22369c056565'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.734059 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24188593926b43ff9d3eec6805c8e359', 'request_id': 'req-4c131ee0-64e8-44df-9b8a-22369c056565'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.740281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24188593926b43ff9d3eec6805c8e359', 'request_id': 'req-4c131ee0-64e8-44df-9b8a-22369c056565'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.755534 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] The token cache is being invalidate because project 24188593926b43ff9d3eec6805c8e359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:27.755827 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.756513 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.758120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1443/2892] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:27 2022] DELETE /identity/v3/projects/24188593926b43ff9d3eec6805c8e359 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:27.762645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:27.772768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a506f45-6257-4760-aaf5-fda8ec6d7041 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:27.834923 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:27.835904 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:27.836177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:27.836491 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] PATH_INFO: `/v3/users/791a7cb1ae2b4b22aee3021fd26c543c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:27.837697 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] RBAC: Authorizing `identity:delete_user(user_id=791a7cb1ae2b4b22aee3021fd26c543c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:27.839637 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:27.891823 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791a7cb1ae2b4b22aee3021fd26c543c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.900807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791a7cb1ae2b4b22aee3021fd26c543c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.913085 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791a7cb1ae2b4b22aee3021fd26c543c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.913724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791a7cb1ae2b4b22aee3021fd26c543c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.921459 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79da85a8-f74a-4a97-99a8-8742ffec063f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791a7cb1ae2b4b22aee3021fd26c543c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:27.928036 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1450/2893] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:27 2022] DELETE /identity/v3/users/791a7cb1ae2b4b22aee3021fd26c543c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:28.219217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:28.220861 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:28.236881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c131ee0-64e8-44df-9b8a-22369c056565 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:28.300052 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:28.300728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.301011 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.301281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] PATH_INFO: `/v3/projects/5685a1babdfc42b1b1bb71074853737c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.302503 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5685a1babdfc42b1b1bb71074853737c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:28.304493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:28.355690 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5685a1babdfc42b1b1bb71074853737c', 'request_id': 'req-1e07d5be-3558-4a47-9078-96f4d3299dba'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:28.362461 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5685a1babdfc42b1b1bb71074853737c', 'request_id': 'req-1e07d5be-3558-4a47-9078-96f4d3299dba'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:28.369147 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5685a1babdfc42b1b1bb71074853737c', 'request_id': 'req-1e07d5be-3558-4a47-9078-96f4d3299dba'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:28.383730 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] The token cache is being invalidate because project 5685a1babdfc42b1b1bb71074853737c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:28.384017 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:28.384673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1e07d5be-3558-4a47-9078-96f4d3299dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:28.386466 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1444/2894] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:28 2022] DELETE /identity/v3/projects/5685a1babdfc42b1b1bb71074853737c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:28.391413 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01821346-f7f6-486c-a2ba-82f534fd0645 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.391887 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01821346-f7f6-486c-a2ba-82f534fd0645 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.391981 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01821346-f7f6-486c-a2ba-82f534fd0645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.412300 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-01821346-f7f6-486c-a2ba-82f534fd0645 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:28.413438 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-01821346-f7f6-486c-a2ba-82f534fd0645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:28.447766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:28.484386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1451/2895] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:28.492378 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69558f-7b0e-4343-97c5-725bc61909b7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.492668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69558f-7b0e-4343-97c5-725bc61909b7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.492965 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69558f-7b0e-4343-97c5-725bc61909b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.515495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3d69558f-7b0e-4343-97c5-725bc61909b7 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:28.516738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3d69558f-7b0e-4343-97c5-725bc61909b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:28.525938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1445/2896] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:28.532090 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01821346-f7f6-486c-a2ba-82f534fd0645 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:28.542884 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01821346-f7f6-486c-a2ba-82f534fd0645 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:28.548964 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:28.549576 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.549837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.550077 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.550745 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:44:28.551041 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:28.552696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:28.569302 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1452/2897] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:44:28.574084 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69558f-7b0e-4343-97c5-725bc61909b7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:28.583791 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69558f-7b0e-4343-97c5-725bc61909b7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:28.590081 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:28.590673 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.590957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.591191 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.591865 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:28.593799 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:28.609697 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:28.628831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1446/2898] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:28 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:28.631709 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:28.643622 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f83728-4811-4dff-9628-9dfd7adee322 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:28.708507 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:28.709039 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.709319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.709592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.710552 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:28.712520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:28.735261 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1453/2899] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:28 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:28.739595 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:28.750240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e7a0f-c60a-4c87-82dc-353e737e1490 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:28.757093 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:28.757967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.758345 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.758711 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.759686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:28.761663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:28.772547 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1447/2900] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:28.776723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:28.787573 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed88954-c519-4b5e-bc1f-650460255e86 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:28.793518 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a23eb6-704e-412d-b0b9-3935e8eac7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:28.794125 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23eb6-704e-412d-b0b9-3935e8eac7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.794404 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23eb6-704e-412d-b0b9-3935e8eac7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.794673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23eb6-704e-412d-b0b9-3935e8eac7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.795312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a23eb6-704e-412d-b0b9-3935e8eac7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:28.796971 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a23eb6-704e-412d-b0b9-3935e8eac7ab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:28.808460 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1454/2901] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:28.813426 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:28.827379 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49604b9-a5e4-4cdb-a877-998c490e7fd6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:28.833385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:28.834039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.834315 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.834589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] PATH_INFO: `/v3/projects/c8ede533e9884d2d8a6ad0d9b21b964b/users/fbbe82f27e014bda91afb18c2ba7a529/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.850530 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ede533e9884d2d8a6ad0d9b21b964b, user_id=fbbe82f27e014bda91afb18c2ba7a529, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:28.853228 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:28.870332 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1448/2902] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:28 2022] PUT /identity/v3/projects/c8ede533e9884d2d8a6ad0d9b21b964b/users/fbbe82f27e014bda91afb18c2ba7a529/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:44:28.882520 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227cfda-b8ea-4e8a-a761-463ded5689fc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:28.882930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227cfda-b8ea-4e8a-a761-463ded5689fc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:28.883474 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227cfda-b8ea-4e8a-a761-463ded5689fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:28.912915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7227cfda-b8ea-4e8a-a761-463ded5689fc None None] MFA Rules not processed for user `fbbe82f27e014bda91afb18c2ba7a529`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:28.914117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7227cfda-b8ea-4e8a-a761-463ded5689fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:28.980461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1455/2903] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:28.985036 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.053659 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e732b55-5373-4e65-b799-ea1128612ace admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.059425 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.060075 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.060359 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.060627 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.061276 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.063190 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.093039 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1449/2904] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:28 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:29.097389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227cfda-b8ea-4e8a-a761-463ded5689fc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.108640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227cfda-b8ea-4e8a-a761-463ded5689fc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.135322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:29.168186 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.168825 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.169099 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.169367 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.169985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.171910 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.194357 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1456/2905] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:29 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:29.198716 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.209457 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a179b2-dce1-41db-ba60-43468e761730 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.214739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.215333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.215608 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.215874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.216591 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.218166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.228804 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1450/2906] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:44:29.233072 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.243437 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58df7e-8de8-4452-8457-bbc429c9c318 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.248480 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.249123 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.249405 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.249680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] PATH_INFO: `/v3/projects/c8ae36f60a364dc3b2c4ed2e4254c95d/users/bede32ad8e464a36aebffeea5d12b3f2/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.266132 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ae36f60a364dc3b2c4ed2e4254c95d, user_id=bede32ad8e464a36aebffeea5d12b3f2, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.268878 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.285315 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1457/2907] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:29 2022] PUT /identity/v3/projects/c8ae36f60a364dc3b2c4ed2e4254c95d/users/bede32ad8e464a36aebffeea5d12b3f2/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:29.289508 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.300185 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd9737-7799-479e-a0a6-a7c7c5b0e68f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.362689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d2459-bbfb-42e3-aba0-6aead842883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.363459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2459-bbfb-42e3-aba0-6aead842883c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.363739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2459-bbfb-42e3-aba0-6aead842883c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.364009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d2459-bbfb-42e3-aba0-6aead842883c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.364812 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d2459-bbfb-42e3-aba0-6aead842883c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.367377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d2459-bbfb-42e3-aba0-6aead842883c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.378129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1451/2908] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:29.382662 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.394615 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea200917-4f00-4294-a123-32e43e7752da admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.399696 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.400484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.400928 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.401209 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] PATH_INFO: `/v3/projects/c8ae36f60a364dc3b2c4ed2e4254c95d/users/bede32ad8e464a36aebffeea5d12b3f2/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.403268 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ae36f60a364dc3b2c4ed2e4254c95d, user_id=bede32ad8e464a36aebffeea5d12b3f2, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.405982 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.416589 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1458/2909] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:29 2022] PUT /identity/v3/projects/c8ae36f60a364dc3b2c4ed2e4254c95d/users/bede32ad8e464a36aebffeea5d12b3f2/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:29.429160 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1693-42b2-475e-8bdb-3f13ef6be91f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.429443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1693-42b2-475e-8bdb-3f13ef6be91f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.429720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f1693-42b2-475e-8bdb-3f13ef6be91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.463459 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9d6f1693-42b2-475e-8bdb-3f13ef6be91f None None] MFA Rules not processed for user `bede32ad8e464a36aebffeea5d12b3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:29.464853 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9d6f1693-42b2-475e-8bdb-3f13ef6be91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:29.481959 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:29.543252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1452/2910] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:44:29.554975 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.557233 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:29.619774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40620af-2c7f-4a9a-a749-18031bad84e6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.623446 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.624080 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.624382 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.624656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.635895 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.637448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.647251 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1459/2911] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:29.739580 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1693-42b2-475e-8bdb-3f13ef6be91f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:29.750382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f1693-42b2-475e-8bdb-3f13ef6be91f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:29.754521 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f9047-7e07-48de-b5bd-03bb8e7b08f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:29.755125 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9047-7e07-48de-b5bd-03bb8e7b08f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:29.755403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9047-7e07-48de-b5bd-03bb8e7b08f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:29.755668 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9047-7e07-48de-b5bd-03bb8e7b08f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:29.820295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f9047-7e07-48de-b5bd-03bb8e7b08f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:29.822389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f9047-7e07-48de-b5bd-03bb8e7b08f9 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:29.836321 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1453/2912] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:34.868989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:34.944384 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6436cf2-c08e-4b34-ba9a-c0613299a29c service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:34.949556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:34.950217 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:34.950489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:34.950760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] PATH_INFO: `/v3/users/fbbe82f27e014bda91afb18c2ba7a529` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:34.951902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbe82f27e014bda91afb18c2ba7a529)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:34.953866 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:34.985139 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:35.006149 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbe82f27e014bda91afb18c2ba7a529'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.015242 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbe82f27e014bda91afb18c2ba7a529'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.027525 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbe82f27e014bda91afb18c2ba7a529'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.028295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbe82f27e014bda91afb18c2ba7a529'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.035328 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbe82f27e014bda91afb18c2ba7a529'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.041548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1460/2913] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:34 2022] DELETE /identity/v3/users/fbbe82f27e014bda91afb18c2ba7a529 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:35.046473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:35.046950 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:35.047038 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:35.069331 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:35.070563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:35.136967 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1454/2914] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:44:35.149277 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:35.151782 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:35.212189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454cad9-41ff-49ad-803c-7d2c11bcd7f2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:35.217522 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:35.218137 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:35.218419 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:35.218691 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:35.228448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:35.229891 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:35.240496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1461/2915] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:35.550948 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:35.552477 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:35.566586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbc23a-df87-4ce6-ba93-2f3e945727c6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:35.572110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:35.572810 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:35.573057 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:35.573326 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] PATH_INFO: `/v3/projects/c8ede533e9884d2d8a6ad0d9b21b964b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:35.575002 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ede533e9884d2d8a6ad0d9b21b964b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:35.577085 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:35.625992 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:35.627375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ede533e9884d2d8a6ad0d9b21b964b', 'request_id': 'req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.634297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ede533e9884d2d8a6ad0d9b21b964b', 'request_id': 'req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.640578 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ede533e9884d2d8a6ad0d9b21b964b', 'request_id': 'req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.655199 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] The token cache is being invalidate because project c8ede533e9884d2d8a6ad0d9b21b964b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:35.655488 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.656126 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.657675 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1455/2916] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:35 2022] DELETE /identity/v3/projects/c8ede533e9884d2d8a6ad0d9b21b964b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:35.661974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:35.672319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c065484-dd0c-4027-a910-1245e1f37395 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:35.731428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:35.732057 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:35.732370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:35.732646 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] PATH_INFO: `/v3/users/bede32ad8e464a36aebffeea5d12b3f2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:35.733838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bede32ad8e464a36aebffeea5d12b3f2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:35.735790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:35.785043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bede32ad8e464a36aebffeea5d12b3f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.786733 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:35.794047 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bede32ad8e464a36aebffeea5d12b3f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.808189 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bede32ad8e464a36aebffeea5d12b3f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.808915 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bede32ad8e464a36aebffeea5d12b3f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.816670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-892121e2-284b-42bf-8c70-6770649fbfe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bede32ad8e464a36aebffeea5d12b3f2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:35.823324 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1462/2917] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:35 2022] DELETE /identity/v3/users/bede32ad8e464a36aebffeea5d12b3f2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:36.108076 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.109720 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:36.124910 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eaf44-f4b8-488f-898e-96915f5cfdd1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.195721 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.196394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.196672 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.196946 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] PATH_INFO: `/v3/projects/c8ae36f60a364dc3b2c4ed2e4254c95d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.198224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ae36f60a364dc3b2c4ed2e4254c95d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.200435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:36.252686 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ae36f60a364dc3b2c4ed2e4254c95d', 'request_id': 'req-dec04579-aa0f-46f5-8e8e-91b6ae44232e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:36.258474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ae36f60a364dc3b2c4ed2e4254c95d', 'request_id': 'req-dec04579-aa0f-46f5-8e8e-91b6ae44232e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:36.266016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ae36f60a364dc3b2c4ed2e4254c95d', 'request_id': 'req-dec04579-aa0f-46f5-8e8e-91b6ae44232e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:36.281745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] The token cache is being invalidate because project c8ae36f60a364dc3b2c4ed2e4254c95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:36.282028 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:36.282684 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dec04579-aa0f-46f5-8e8e-91b6ae44232e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:36.284333 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1456/2918] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:36 2022] DELETE /identity/v3/projects/c8ae36f60a364dc3b2c4ed2e4254c95d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:36.289532 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f58e-62bc-43ce-9e03-50210f497048 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.290025 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f58e-62bc-43ce-9e03-50210f497048 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.290115 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854f58e-62bc-43ce-9e03-50210f497048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.310717 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e854f58e-62bc-43ce-9e03-50210f497048 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:36.311909 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e854f58e-62bc-43ce-9e03-50210f497048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:36.380326 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1463/2919] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:44:36.388548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5599a-b682-4ce8-93eb-bd8693ade3cb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.388831 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5599a-b682-4ce8-93eb-bd8693ade3cb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.389103 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5599a-b682-4ce8-93eb-bd8693ade3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.409360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a7f5599a-b682-4ce8-93eb-bd8693ade3cb None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:36.410561 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a7f5599a-b682-4ce8-93eb-bd8693ade3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:36.420938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1457/2920] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:36.425496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f58e-62bc-43ce-9e03-50210f497048 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.436544 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854f58e-62bc-43ce-9e03-50210f497048 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.441621 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.442254 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.442540 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.442838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.443478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:44:36.443807 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.445433 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:36.457724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1464/2921] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:44:36.462904 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5599a-b682-4ce8-93eb-bd8693ade3cb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.472859 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5599a-b682-4ce8-93eb-bd8693ade3cb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.478280 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.478872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.479148 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.479413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.480035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.482010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:36.513252 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1458/2922] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:36 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:36.519443 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.533206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d5e76-e889-485a-ad5d-c1da6703eadf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.598386 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.599024 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.599303 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.599592 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.600258 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.602195 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:36.625743 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1465/2923] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:36 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:36.630133 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.641217 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f30733-adcd-441d-acd8-3ad5a070fe84 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.646253 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.646887 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.647164 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.647434 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.648077 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.649752 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:36.660548 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1459/2924] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:36.664800 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.675319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe68f6c-5b8a-42ab-9336-007dcdfe7ee2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.680347 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6ac76-de04-4ea2-9651-2922a3bb540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.680952 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6ac76-de04-4ea2-9651-2922a3bb540e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.681229 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6ac76-de04-4ea2-9651-2922a3bb540e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.681495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6ac76-de04-4ea2-9651-2922a3bb540e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.682127 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6ac76-de04-4ea2-9651-2922a3bb540e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.683744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6ac76-de04-4ea2-9651-2922a3bb540e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:36.695034 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1466/2925] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:36.699360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.711109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ca1f5-bd5b-4dc2-b239-2dc5f89d9e54 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.721874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.723403 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.724031 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.724737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] PATH_INFO: `/v3/projects/a8aa50f715a74338b839581155e0600f/users/131f42d4110347efbdb5c66108c95586/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.752789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8aa50f715a74338b839581155e0600f, user_id=131f42d4110347efbdb5c66108c95586, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.757206 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:36.776803 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1460/2926] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:36 2022] PUT /identity/v3/projects/a8aa50f715a74338b839581155e0600f/users/131f42d4110347efbdb5c66108c95586/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:36.793103 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067625-028c-41ec-bdb3-94bfe186001c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.793758 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067625-028c-41ec-bdb3-94bfe186001c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.794038 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067625-028c-41ec-bdb3-94bfe186001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.824780 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-46067625-028c-41ec-bdb3-94bfe186001c None None] MFA Rules not processed for user `131f42d4110347efbdb5c66108c95586`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:36.826093 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-46067625-028c-41ec-bdb3-94bfe186001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:36.847617 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:36.898829 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1467/2927] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:36.903589 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:36.908907 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:36.975495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec1b23-3b87-48d9-b714-08bbde9ef7da admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:36.981102 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:36.981713 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:36.981997 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:36.982269 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:36.982897 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:36.984856 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.013110 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1461/2928] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:36 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:37.017393 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46067625-028c-41ec-bdb3-94bfe186001c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.029301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46067625-028c-41ec-bdb3-94bfe186001c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.089935 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.090649 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.090950 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.091255 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.091894 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.093953 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.117409 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1468/2929] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:37 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:37.122101 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.132847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8ec12-9c07-41f8-8213-2f792cbefeb1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.137956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.138555 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.138837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.139109 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.139755 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.141382 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.152215 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1462/2930] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:37.156471 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.166901 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e9f6d-e30d-4cee-8438-fed46b707ba5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.172078 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9f6e0-5c39-4646-94dc-d87a8c83cc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.172751 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f6e0-5c39-4646-94dc-d87a8c83cc9a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.173033 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f6e0-5c39-4646-94dc-d87a8c83cc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.173312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9f6e0-5c39-4646-94dc-d87a8c83cc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.173946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9f6e0-5c39-4646-94dc-d87a8c83cc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.175556 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9f6e0-5c39-4646-94dc-d87a8c83cc9a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.186120 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1469/2931] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:44:37.190311 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.200610 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82705663-7c42-44c6-8e1d-7bf556f9a459 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.206216 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.206866 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.207144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.207413 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] PATH_INFO: `/v3/projects/f12f0a7eb7bc4591a1a826d7d2c8d516/users/35b42b9f6e894e69a5db437fb795df1d/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.222396 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12f0a7eb7bc4591a1a826d7d2c8d516, user_id=35b42b9f6e894e69a5db437fb795df1d, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.225127 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.241427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1463/2932] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:37 2022] PUT /identity/v3/projects/f12f0a7eb7bc4591a1a826d7d2c8d516/users/35b42b9f6e894e69a5db437fb795df1d/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:37.252984 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d75b9b-cb66-4ff4-bf1b-4a1e17c7a475 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.253356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d75b9b-cb66-4ff4-bf1b-4a1e17c7a475 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.253775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d75b9b-cb66-4ff4-bf1b-4a1e17c7a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.286485 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-44d75b9b-cb66-4ff4-bf1b-4a1e17c7a475 None None] MFA Rules not processed for user `35b42b9f6e894e69a5db437fb795df1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:37.287641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44d75b9b-cb66-4ff4-bf1b-4a1e17c7a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:37.356932 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1470/2933] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:37.361846 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.432999 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923cec6-c0c8-4b98-ad5c-9fecce16dae7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.438843 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.439451 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.439725 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.440009 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.440661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.442569 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.476611 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1464/2934] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:37 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:37.481068 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d75b9b-cb66-4ff4-bf1b-4a1e17c7a475 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.492598 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d75b9b-cb66-4ff4-bf1b-4a1e17c7a475 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.521320 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:37.555087 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.555624 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.555903 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.556192 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.556827 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.558775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.582439 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1471/2935] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:37 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:37.588593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.599350 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb80d57-c079-4d43-98ff-e28535d8a540 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.605435 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.607010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.607295 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.607566 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.608270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.609871 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.620981 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1465/2936] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:37.625333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.635411 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66714ecc-e641-4261-8954-d1566a1fdbdf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.640853 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.641505 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.641788 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.642061 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] PATH_INFO: `/v3/projects/f22dc2baa1ad4123919433ea4e8d985a/users/d6b02e97408e40b9a613043fad96a898/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.656799 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22dc2baa1ad4123919433ea4e8d985a, user_id=d6b02e97408e40b9a613043fad96a898, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.659434 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.682150 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1472/2937] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:37 2022] PUT /identity/v3/projects/f22dc2baa1ad4123919433ea4e8d985a/users/d6b02e97408e40b9a613043fad96a898/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:37.686392 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.699971 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378507de-1d80-4c51-934c-8ab9e831b7a5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.727150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:37.763918 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c53a7-0e08-45c4-bcb0-4ac59d6acce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.764709 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c53a7-0e08-45c4-bcb0-4ac59d6acce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.764984 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c53a7-0e08-45c4-bcb0-4ac59d6acce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.765283 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c53a7-0e08-45c4-bcb0-4ac59d6acce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.765917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c53a7-0e08-45c4-bcb0-4ac59d6acce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.767498 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c53a7-0e08-45c4-bcb0-4ac59d6acce7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.777362 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1466/2938] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:37.781610 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.791873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a36147-48f1-4927-8377-18ba6dc27b7a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:37.796771 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:37.797422 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.797700 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.797966 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] PATH_INFO: `/v3/projects/f22dc2baa1ad4123919433ea4e8d985a/users/d6b02e97408e40b9a613043fad96a898/roles/d6ba822e8c4c4402961c6db999ff6fd1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.799902 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22dc2baa1ad4123919433ea4e8d985a, user_id=d6b02e97408e40b9a613043fad96a898, role_id=d6ba822e8c4c4402961c6db999ff6fd1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:37.802572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:37.813269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1473/2939] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:37 2022] PUT /identity/v3/projects/f22dc2baa1ad4123919433ea4e8d985a/users/d6b02e97408e40b9a613043fad96a898/roles/d6ba822e8c4c4402961c6db999ff6fd1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 05 17:44:37.823314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:37.823623 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:37.823900 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:37.853600 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] MFA Rules not processed for user `d6b02e97408e40b9a613043fad96a898`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:37.854798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:37.927175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1467/2940] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:37.940574 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:37.942388 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:38.009992 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fc99f-7de1-4acf-bc29-fea9154e803c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:38.014529 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:38.015152 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:38.015453 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:38.015723 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:38.073611 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:38.075166 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:38.084671 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1474/2941] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:39.620070 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:39.621642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:39.635375 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff75a36-9dcd-4965-94e4-af6a7035474f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:39.640055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:39.640698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:39.640976 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:39.641246 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:39.710367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:39.711795 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:39.721538 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1468/2942] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:40.309519 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:40.320124 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729cbd7-d584-44b4-82fd-cf30f250b38c service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:40.324484 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a399faf-29c6-41d6-b8f3-83f1bdb439d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:40.325097 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a399faf-29c6-41d6-b8f3-83f1bdb439d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:40.325375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a399faf-29c6-41d6-b8f3-83f1bdb439d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:40.325644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a399faf-29c6-41d6-b8f3-83f1bdb439d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:40.335806 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a399faf-29c6-41d6-b8f3-83f1bdb439d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:40.337327 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a399faf-29c6-41d6-b8f3-83f1bdb439d7 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:40.347001 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1475/2943] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:44.332422 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:44.384043 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:44.402006 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986a7c7-ec5e-4da2-bcf6-3509102a340a service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:44.407388 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:44.408025 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:44.408343 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:44.408618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] PATH_INFO: `/v3/users/131f42d4110347efbdb5c66108c95586` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:44.410000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=131f42d4110347efbdb5c66108c95586)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:44.411947 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:44.466774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131f42d4110347efbdb5c66108c95586'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:44.467473 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131f42d4110347efbdb5c66108c95586'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:44.476917 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131f42d4110347efbdb5c66108c95586'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:44.489565 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131f42d4110347efbdb5c66108c95586'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:44.490250 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131f42d4110347efbdb5c66108c95586'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:44.503144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1469/2944] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:44 2022] DELETE /identity/v3/users/131f42d4110347efbdb5c66108c95586 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:44.507941 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:44.508451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:44.508538 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:44.529725 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:44.530973 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:44.550509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:44.596470 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1476/2945] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:44.606408 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:44.608035 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:44.668175 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5932c-f951-4c29-b3fc-140087c6c553 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:44.672107 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:44.672745 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:44.673016 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:44.673286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:44.682632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:44.684065 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:44.694138 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1470/2946] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:44.985974 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:44.987469 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:45.000929 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b02450-4326-4cf8-8c28-a73fa0899179 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:45.006260 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:45.006944 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:45.007219 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:45.007461 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] PATH_INFO: `/v3/projects/a8aa50f715a74338b839581155e0600f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:45.008783 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8aa50f715a74338b839581155e0600f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:45.010872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:45.059475 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8aa50f715a74338b839581155e0600f', 'request_id': 'req-2a3a39a4-002c-4181-b5f6-e3448babc750'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.065732 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8aa50f715a74338b839581155e0600f', 'request_id': 'req-2a3a39a4-002c-4181-b5f6-e3448babc750'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.071575 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8aa50f715a74338b839581155e0600f', 'request_id': 'req-2a3a39a4-002c-4181-b5f6-e3448babc750'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.085970 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] The token cache is being invalidate because project a8aa50f715a74338b839581155e0600f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:45.086259 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.086867 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.088385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1477/2947] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:44 2022] DELETE /identity/v3/projects/a8aa50f715a74338b839581155e0600f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:45.092515 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:45.102303 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bac54-09d3-4c84-a6c9-e66e3f9214e3 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:45.159647 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:45.160278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:45.160558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:45.160829 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] PATH_INFO: `/v3/users/35b42b9f6e894e69a5db437fb795df1d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:45.161957 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b42b9f6e894e69a5db437fb795df1d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:45.163895 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:45.174550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:45.215923 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b42b9f6e894e69a5db437fb795df1d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.216663 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b42b9f6e894e69a5db437fb795df1d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.225305 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b42b9f6e894e69a5db437fb795df1d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.238088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b42b9f6e894e69a5db437fb795df1d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.238874 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b42b9f6e894e69a5db437fb795df1d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.251983 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1471/2948] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:45 2022] DELETE /identity/v3/users/35b42b9f6e894e69a5db437fb795df1d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:45.539163 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:45.541288 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:45.555653 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a39a4-002c-4181-b5f6-e3448babc750 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:45.615086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:45.615639 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:45.615922 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:45.616224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] PATH_INFO: `/v3/projects/f12f0a7eb7bc4591a1a826d7d2c8d516` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:45.617365 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12f0a7eb7bc4591a1a826d7d2c8d516)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:45.619295 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:45.667091 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:45.668333 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12f0a7eb7bc4591a1a826d7d2c8d516', 'request_id': 'req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.675531 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12f0a7eb7bc4591a1a826d7d2c8d516', 'request_id': 'req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.682122 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12f0a7eb7bc4591a1a826d7d2c8d516', 'request_id': 'req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.695352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] The token cache is being invalidate because project f12f0a7eb7bc4591a1a826d7d2c8d516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:45.695644 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.696298 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.697818 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1478/2949] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:45 2022] DELETE /identity/v3/projects/f12f0a7eb7bc4591a1a826d7d2c8d516 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:45.701933 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:45.703351 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:45.717773 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f04955-0def-494b-858f-d360ec1da676 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:45.778632 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:45.779270 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:45.779550 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:45.779825 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] PATH_INFO: `/v3/users/d6b02e97408e40b9a613043fad96a898` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:45.781018 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b02e97408e40b9a613043fad96a898)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:45.783689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:45.836231 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b02e97408e40b9a613043fad96a898'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.836881 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b02e97408e40b9a613043fad96a898'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.845754 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b02e97408e40b9a613043fad96a898'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.858787 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b02e97408e40b9a613043fad96a898'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.859421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8bde1acd-ad89-44fd-a34e-a3e1bc99f133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b02e97408e40b9a613043fad96a898'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:45.871821 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1472/2950] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:44:45 2022] DELETE /identity/v3/users/d6b02e97408e40b9a613043fad96a898 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:46.150632 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.152116 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:46.170967 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fe3f4-ddf3-4ba5-8c56-c6842f241754 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.232735 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.233376 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.233661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.233924 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] PATH_INFO: `/v3/projects/f22dc2baa1ad4123919433ea4e8d985a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.235059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22dc2baa1ad4123919433ea4e8d985a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.237070 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.291606 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22dc2baa1ad4123919433ea4e8d985a', 'request_id': 'req-35bc9351-cd31-4dca-94f8-a13dbbde8d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:46.297985 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22dc2baa1ad4123919433ea4e8d985a', 'request_id': 'req-35bc9351-cd31-4dca-94f8-a13dbbde8d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:46.303672 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22dc2baa1ad4123919433ea4e8d985a', 'request_id': 'req-35bc9351-cd31-4dca-94f8-a13dbbde8d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:46.318067 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] The token cache is being invalidate because project f22dc2baa1ad4123919433ea4e8d985a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:44:46.318354 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:46.318998 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35bc9351-cd31-4dca-94f8-a13dbbde8d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:44:46.320533 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1479/2951] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:44:46 2022] DELETE /identity/v3/projects/f22dc2baa1ad4123919433ea4e8d985a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:46.325489 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762f216-8db1-41ed-abb9-38c29fc281cf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.325774 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762f216-8db1-41ed-abb9-38c29fc281cf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.326054 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762f216-8db1-41ed-abb9-38c29fc281cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.340742 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:46.347296 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c762f216-8db1-41ed-abb9-38c29fc281cf None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:46.349427 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c762f216-8db1-41ed-abb9-38c29fc281cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:46.410747 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1473/2952] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 05 17:44:46.418927 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cb60-7048-4596-9fbd-3a8fac7c5fd3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.419208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cb60-7048-4596-9fbd-3a8fac7c5fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.419476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38cb60-7048-4596-9fbd-3a8fac7c5fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.440198 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8d38cb60-7048-4596-9fbd-3a8fac7c5fd3 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:46.441501 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8d38cb60-7048-4596-9fbd-3a8fac7c5fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:46.450486 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1480/2953] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:46.454912 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762f216-8db1-41ed-abb9-38c29fc281cf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.466744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762f216-8db1-41ed-abb9-38c29fc281cf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.473499 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.474108 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.474385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.474653 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.475324 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:44:46.475618 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.477267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.490443 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1474/2954] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:44:46.495098 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38cb60-7048-4596-9fbd-3a8fac7c5fd3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.505597 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38cb60-7048-4596-9fbd-3a8fac7c5fd3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.511667 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.512312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.512590 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.512862 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.513492 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.515427 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.547438 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1481/2955] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:44:46 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:46.552360 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.562469 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b6263-d617-4a08-8466-3530f3ee1741 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.622230 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.622837 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.623120 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.623394 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.624013 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.625985 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.651525 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1475/2956] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:44:46 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:44:46.655774 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.666583 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f3f25-0b05-4b99-807d-712d6d82d923 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.672248 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.672794 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.673081 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.673352 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.674005 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.675850 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.686141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1482/2957] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:46.690421 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.700528 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bfe64-c69e-40f2-8215-d114f733eb55 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.705144 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ce69f-76c3-4237-8ce3-a991cdea7747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.705800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ce69f-76c3-4237-8ce3-a991cdea7747 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.706026 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ce69f-76c3-4237-8ce3-a991cdea7747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.706297 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ce69f-76c3-4237-8ce3-a991cdea7747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.706940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ce69f-76c3-4237-8ce3-a991cdea7747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.708656 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ce69f-76c3-4237-8ce3-a991cdea7747 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.719061 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1476/2958] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:44:46.723674 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.734432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c6386-7465-43ee-ba7b-601e2adb24de admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.739645 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.740351 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.740656 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.740934 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] PATH_INFO: `/v3/projects/0fb02bbe669640a0997c8aaa624caa8b/users/9ef60d25b1074ebaa30d99ef367594de/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.757054 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb02bbe669640a0997c8aaa624caa8b, user_id=9ef60d25b1074ebaa30d99ef367594de, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.759766 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.775246 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1483/2959] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:44:46 2022] PUT /identity/v3/projects/0fb02bbe669640a0997c8aaa624caa8b/users/9ef60d25b1074ebaa30d99ef367594de/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:44:46.789001 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.789281 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.789549 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.820988 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] MFA Rules not processed for user `9ef60d25b1074ebaa30d99ef367594de`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:44:46.822214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:46.892587 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1477/2960] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:44:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:44:46.908489 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:44:46.910682 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:44:46.917086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:46.971945 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d209ad-f447-46aa-8619-aac271af46f4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:44:46.976117 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05a016553950460dbb736ced2cae4f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:44:46.976760 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:44:46.977045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:44:46.977322 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:44:46.985871 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:44:46.987350 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:44:46.996287 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1484/2961] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:44:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:44:56.889662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:44:56.992273 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=25 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:45:28.129738 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:28.131174 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:28.189505 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa6f7-7dde-43d1-95c9-6f4d715b95c3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:28.193701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:28.194389 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:28.194677 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:28.195000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:28.196789 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:28.207574 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:28.209204 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:28.218765 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1478/2962] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:45:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:45:28.327342 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:28.343495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149910e-98a7-4460-8f08-516cd581af9b service glance] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:28.348236 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7dc188-46de-449d-a2b6-4f18e89829aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:28.348838 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc188-46de-449d-a2b6-4f18e89829aa service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:28.349110 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc188-46de-449d-a2b6-4f18e89829aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:28.349372 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7dc188-46de-449d-a2b6-4f18e89829aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:28.351370 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-de7dc188-46de-449d-a2b6-4f18e89829aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:28.428548 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dc188-46de-449d-a2b6-4f18e89829aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:28.430121 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7dc188-46de-449d-a2b6-4f18e89829aa service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:28.441155 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1485/2963] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:45:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:45:31.303082 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:31.371604 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0b02-0a88-4ae9-88e4-59d9395cfa4c service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:31.376429 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:31.377083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:31.377367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:31.377628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] PATH_INFO: `/v3/users/9ef60d25b1074ebaa30d99ef367594de` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:31.379184 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef60d25b1074ebaa30d99ef367594de)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:31.381367 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:31.430010 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef60d25b1074ebaa30d99ef367594de'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:31.430743 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef60d25b1074ebaa30d99ef367594de'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:31.439702 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef60d25b1074ebaa30d99ef367594de'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:31.451776 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef60d25b1074ebaa30d99ef367594de'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:31.452478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef60d25b1074ebaa30d99ef367594de'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:31.466628 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1479/2964] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:45:31 2022] DELETE /identity/v3/users/9ef60d25b1074ebaa30d99ef367594de => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:45:31.471256 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:31.471729 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:31.471844 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:31.492863 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:45:31.494096 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:31.553976 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1486/2965] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:45:31.563764 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:31.565264 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:31.592045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:45:31.627817 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2ba99-ee88-4868-b4bd-11d1b9be899a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:31.631941 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448a924-d346-4373-87b3-436cbb1be74d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:31.632577 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a924-d346-4373-87b3-436cbb1be74d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:31.632855 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a924-d346-4373-87b3-436cbb1be74d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:31.633129 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a924-d346-4373-87b3-436cbb1be74d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:31.642800 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448a924-d346-4373-87b3-436cbb1be74d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:31.644272 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448a924-d346-4373-87b3-436cbb1be74d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:31.654240 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1480/2966] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:45:31.939366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:31.942182 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:31.954340 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7dd51-8093-42f2-996e-8dc5a460d257 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:31.960128 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:31.960808 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:31.961086 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:31.961356 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] PATH_INFO: `/v3/projects/0fb02bbe669640a0997c8aaa624caa8b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:31.962569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb02bbe669640a0997c8aaa624caa8b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:31.964553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.015572 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb02bbe669640a0997c8aaa624caa8b', 'request_id': 'req-3463a751-cc45-459d-a1f0-0a212f1afc09'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.022224 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb02bbe669640a0997c8aaa624caa8b', 'request_id': 'req-3463a751-cc45-459d-a1f0-0a212f1afc09'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.027633 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb02bbe669640a0997c8aaa624caa8b', 'request_id': 'req-3463a751-cc45-459d-a1f0-0a212f1afc09'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.042290 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] The token cache is being invalidate because project 0fb02bbe669640a0997c8aaa624caa8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:45:32.042582 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.043239 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3463a751-cc45-459d-a1f0-0a212f1afc09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.044817 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1487/2967] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:45:31 2022] DELETE /identity/v3/projects/0fb02bbe669640a0997c8aaa624caa8b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:45:32.050405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277742f-b362-456e-a979-7267cc091bac None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.050739 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277742f-b362-456e-a979-7267cc091bac None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.051116 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277742f-b362-456e-a979-7267cc091bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.071518 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6277742f-b362-456e-a979-7267cc091bac None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:45:32.072693 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6277742f-b362-456e-a979-7267cc091bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:32.140179 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1481/2968] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:45:32.148020 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5ee-107d-4e04-bc37-0de040306897 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.148375 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5ee-107d-4e04-bc37-0de040306897 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.148660 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb5ee-107d-4e04-bc37-0de040306897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.169640 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a5ffb5ee-107d-4e04-bc37-0de040306897 None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:45:32.170837 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a5ffb5ee-107d-4e04-bc37-0de040306897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:32.180448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1488/2969] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:45:32.185042 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277742f-b362-456e-a979-7267cc091bac None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.195669 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277742f-b362-456e-a979-7267cc091bac None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.201798 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.202385 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.202796 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.203072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.203698 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 05 17:45:32.204000 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.205625 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.224406 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1482/2970] 10.222.0.23 () {58 vars in 1153 bytes} [Mon Sep 5 17:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 05 17:45:32.229016 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb5ee-107d-4e04-bc37-0de040306897 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.233569 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:45:32.241199 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb5ee-107d-4e04-bc37-0de040306897 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.248004 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.248819 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.249105 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.249385 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.250045 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.252432 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.282641 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1489/2971] 10.222.0.23 () {60 vars in 1153 bytes} [Mon Sep 5 17:45:32 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:45:32.287486 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.298161 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ab895-9437-475a-b32e-48b3e6377c83 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.362901 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.363737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.364088 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.364309 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.364952 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.366906 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.392655 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1483/2972] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Sep 5 17:45:32 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 05 17:45:32.396979 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.408265 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e14da7b-4632-4779-b68e-d5c89f1fec03 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.413159 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.413763 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.414043 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.414314 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.414930 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.416607 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.427565 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1490/2973] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 05 17:45:32.432214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.445130 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b38f38c-1577-40fd-b707-39a12d6d484f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.450337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cc5f0-4c9e-468a-a5e6-dbfb3f718685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.450930 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cc5f0-4c9e-468a-a5e6-dbfb3f718685 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.451227 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cc5f0-4c9e-468a-a5e6-dbfb3f718685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.451504 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cc5f0-4c9e-468a-a5e6-dbfb3f718685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.452140 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cc5f0-4c9e-468a-a5e6-dbfb3f718685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.453770 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cc5f0-4c9e-468a-a5e6-dbfb3f718685 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.465551 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1484/2974] 10.222.0.23 () {58 vars in 1122 bytes} [Mon Sep 5 17:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 05 17:45:32.469739 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.480790 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25abb1d0-bd20-44c2-a3a9-9bb34d2af4c3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.485793 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.486448 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.486724 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.487008 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] PATH_INFO: `/v3/projects/77bb65d1855e4ddea0ea96ddaa06bf2e/users/e2d331d3bfed4634808da59339659425/roles/e68472fdebed4f58b495dc466408a9e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.504366 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bb65d1855e4ddea0ea96ddaa06bf2e, user_id=e2d331d3bfed4634808da59339659425, role_id=e68472fdebed4f58b495dc466408a9e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.506946 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.523169 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1491/2975] 10.222.0.23 () {60 vars in 1483 bytes} [Mon Sep 5 17:45:32 2022] PUT /identity/v3/projects/77bb65d1855e4ddea0ea96ddaa06bf2e/users/e2d331d3bfed4634808da59339659425/roles/e68472fdebed4f58b495dc466408a9e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:45:32.534095 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41796796-1ea7-43e2-9fef-373510d82ae7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.534366 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41796796-1ea7-43e2-9fef-373510d82ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.534661 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41796796-1ea7-43e2-9fef-373510d82ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.563115 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-41796796-1ea7-43e2-9fef-373510d82ae7 None None] MFA Rules not processed for user `e2d331d3bfed4634808da59339659425`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:45:32.564411 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-41796796-1ea7-43e2-9fef-373510d82ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:32.585478 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:45:32.637662 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1485/2976] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:45:32.649397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.651144 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:32.726959 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807e37a-c3d6-46d8-bd7e-3f1a4949bec8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.731173 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de86d20-bf22-48ce-b856-ef58d51d34a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c795d7c6bc4c519c15f32489c44583', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.731775 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de86d20-bf22-48ce-b856-ef58d51d34a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.732023 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de86d20-bf22-48ce-b856-ef58d51d34a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.732581 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de86d20-bf22-48ce-b856-ef58d51d34a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.743059 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de86d20-bf22-48ce-b856-ef58d51d34a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.745250 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de86d20-bf22-48ce-b856-ef58d51d34a2 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.757563 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1492/2977] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:45:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:45:32.783581 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41796796-1ea7-43e2-9fef-373510d82ae7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:32.851214 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41796796-1ea7-43e2-9fef-373510d82ae7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:32.856634 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:32.857262 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.857553 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.857820 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] PATH_INFO: `/v3/users/e2d331d3bfed4634808da59339659425` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.859083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d331d3bfed4634808da59339659425)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:32.861045 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:32.913017 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d331d3bfed4634808da59339659425'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.913694 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d331d3bfed4634808da59339659425'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.922405 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d331d3bfed4634808da59339659425'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.934993 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d331d3bfed4634808da59339659425'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.935642 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d331d3bfed4634808da59339659425'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:32.948626 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1486/2978] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:45:32 2022] DELETE /identity/v3/users/e2d331d3bfed4634808da59339659425 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:45:32.953206 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:32.953509 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:32.953776 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:32.973773 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] MFA Rules not processed for user `fce1040c54ac43928ab07aa4e6b6d613`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 05 17:45:32.974980 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:33.038428 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1493/2979] 10.222.0.23 () {58 vars in 958 bytes} [Mon Sep 5 17:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 05 17:45:33.048520 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:33.050029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:33.102870 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:45:33.112377 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6629e-e349-4495-a1bd-cd2612d04506 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:33.117072 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:33.117792 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:33.118135 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:33.118474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:33.128015 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:33.129857 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:33.139703 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1487/2980] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:45:33.418702 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:45:33.420308 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:45:33.434445 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df8885-f9ae-4cc0-a0e0-0e49594d539c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:45:33.439648 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:45:33.440527 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:45:33.441032 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:45:33.441208 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] PATH_INFO: `/v3/projects/77bb65d1855e4ddea0ea96ddaa06bf2e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:45:33.442560 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77bb65d1855e4ddea0ea96ddaa06bf2e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:45:33.444744 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:45:33.497670 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:45:33.502593 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77bb65d1855e4ddea0ea96ddaa06bf2e', 'request_id': 'req-73399836-6849-4f11-ad58-aa03a6d80cc3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:33.509680 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77bb65d1855e4ddea0ea96ddaa06bf2e', 'request_id': 'req-73399836-6849-4f11-ad58-aa03a6d80cc3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:33.515177 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77bb65d1855e4ddea0ea96ddaa06bf2e', 'request_id': 'req-73399836-6849-4f11-ad58-aa03a6d80cc3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:33.529205 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] The token cache is being invalidate because project 77bb65d1855e4ddea0ea96ddaa06bf2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:45:33.529495 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:33.530111 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:45:33.531673 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1494/2981] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:45:33 2022] DELETE /identity/v3/projects/77bb65d1855e4ddea0ea96ddaa06bf2e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:45:43.135519 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=8 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:45:43.526042 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:46:19.074701 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:46:19.077267 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:46:19.150449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c034a34-dea3-4988-ba01-97b4e10a96b6 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:46:19.214883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:46:19.215759 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:46:19.216218 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:46:19.216586 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:46:19.219732 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:46:19.237556 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:46:19.239137 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:46:19.249744 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1488/2982] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:46:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:46:29.245620 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=58 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:47:13.926939 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:47:13.929710 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:14.012269 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73399836-6849-4f11-ad58-aa03a6d80cc3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:47:14.016553 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:47:14.017170 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:47:14.017451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:47:14.017718 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:47:14.019536 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:14.085803 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:47:14.087244 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:47:14.096872 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1495/2983] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:47:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:47:14.996005 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:47:14.997955 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:15.092495 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbac3e5-6fd1-40f1-af7f-0581bbf93843 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:47:15.099244 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c30e20ce4d63452095aedb60249f0cce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:47:15.099872 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:47:15.100166 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:47:15.100449 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:47:15.102224 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:15.115593 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:47:15.117286 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:47:15.135073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1489/2984] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:47:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:47:18.938319 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:47:18.948741 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77092dbd-23c8-47c0-86eb-3d8acdd6a54d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:47:18.952830 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcbcd7638cc84cdd87a1eb98f5926425', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:47:18.953451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:47:18.953731 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:47:18.954050 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:47:18.956476 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:19.024843 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:47:19.026312 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:47:19.036397 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1496/2985] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:47:25.128079 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=39 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:47:25.465150 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:47:25.466731 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:25.481156 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f62d8-d899-4a49-81b4-595fcaa7bc71 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:47:25.485640 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:47:25.486278 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:47:25.486558 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:47:25.486830 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] PATH_INFO: `/v3/users/f0a8514ae867426c80200fbc90435c6d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:47:25.488083 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a8514ae867426c80200fbc90435c6d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:47:25.490134 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:47:25.540291 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a8514ae867426c80200fbc90435c6d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.540938 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a8514ae867426c80200fbc90435c6d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.550024 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a8514ae867426c80200fbc90435c6d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.562940 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a8514ae867426c80200fbc90435c6d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.563568 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a8514ae867426c80200fbc90435c6d'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.576826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1490/2986] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:47:25 2022] DELETE /identity/v3/users/f0a8514ae867426c80200fbc90435c6d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:47:25.758661 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:47:25.770687 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cca7a-6ae0-4269-8c7b-e6a72e76af53 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:47:25.812997 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:47:25.838961 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:47:25.839600 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:47:25.839873 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:47:25.840141 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] PATH_INFO: `/v3/projects/668b8711b6fd422b9e58f852c1801569` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:47:25.841496 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=668b8711b6fd422b9e58f852c1801569)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:47:25.843451 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:47:25.893266 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668b8711b6fd422b9e58f852c1801569', 'request_id': 'req-9d1e3bee-c055-4085-9fd0-509a3919be75'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.899499 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668b8711b6fd422b9e58f852c1801569', 'request_id': 'req-9d1e3bee-c055-4085-9fd0-509a3919be75'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.905172 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668b8711b6fd422b9e58f852c1801569', 'request_id': 'req-9d1e3bee-c055-4085-9fd0-509a3919be75'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.918478 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] The token cache is being invalidate because project 668b8711b6fd422b9e58f852c1801569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:47:25.918768 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.919389 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:25.920918 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1497/2987] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:47:25 2022] DELETE /identity/v3/projects/668b8711b6fd422b9e58f852c1801569 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:47:25.925409 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:47:25.926956 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:25.940474 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac52fa3-88ac-429a-a48b-853f3c0b4faf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:47:26.003619 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:47:26.004284 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:47:26.004563 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:47:26.004834 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] PATH_INFO: `/v3/users/0f69ea3b15e944e1832d2230cd1b8bb8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:47:26.006055 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f69ea3b15e944e1832d2230cd1b8bb8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:47:26.007961 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:47:26.057090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f69ea3b15e944e1832d2230cd1b8bb8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.057826 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f69ea3b15e944e1832d2230cd1b8bb8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.067493 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f69ea3b15e944e1832d2230cd1b8bb8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.080635 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f69ea3b15e944e1832d2230cd1b8bb8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.081337 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f69ea3b15e944e1832d2230cd1b8bb8'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.083583 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:47:26.099728 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1491/2988] 10.222.0.23 () {58 vars in 1224 bytes} [Mon Sep 5 17:47:25 2022] DELETE /identity/v3/users/0f69ea3b15e944e1832d2230cd1b8bb8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:47:26.283301 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:47:26.284948 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:47:26.300845 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e3bee-c055-4085-9fd0-509a3919be75 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:47:26.361035 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fce1040c54ac43928ab07aa4e6b6d613', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '831cdea88c914362aff075e5d457eeb4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:47:26.361675 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:47:26.361949 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:47:26.362278 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] PATH_INFO: `/v3/projects/2064744cd17045e9ae2a74670c9f70fb` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:47:26.363770 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2064744cd17045e9ae2a74670c9f70fb)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:47:26.365995 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:47:26.417885 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2064744cd17045e9ae2a74670c9f70fb', 'request_id': 'req-a31c1859-7363-4842-b7bc-e691f620ad89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.424216 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2064744cd17045e9ae2a74670c9f70fb', 'request_id': 'req-a31c1859-7363-4842-b7bc-e691f620ad89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.430450 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2064744cd17045e9ae2a74670c9f70fb', 'request_id': 'req-a31c1859-7363-4842-b7bc-e691f620ad89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.441373 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:47:26.446394 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] The token cache is being invalidate because project 2064744cd17045e9ae2a74670c9f70fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 05 17:47:26.447292 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.448275 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a31c1859-7363-4842-b7bc-e691f620ad89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 05 17:47:26.449897 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1498/2989] 10.222.0.23 () {58 vars in 1233 bytes} [Mon Sep 5 17:47:26 2022] DELETE /identity/v3/projects/2064744cd17045e9ae2a74670c9f70fb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 05 17:47:36.095689 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:47:36.453989 dv-cinder-855964-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 05 17:51:27.058314 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 05 17:51:27.059847 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:51:27.161786 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62a0c4-9db4-4531-98c8-61e54fa041a7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 05 17:51:27.226090 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53981827-81c9-4d79-ae9c-26037450c6e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b0d695705124ac0b5d0a63e70ad45ce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ba6bbf6ebf764fe08027e5e11c0e65c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 05 17:51:27.226987 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981827-81c9-4d79-ae9c-26037450c6e1 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 05 17:51:27.227590 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981827-81c9-4d79-ae9c-26037450c6e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 05 17:51:27.227883 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981827-81c9-4d79-ae9c-26037450c6e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 05 17:51:27.229737 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-53981827-81c9-4d79-ae9c-26037450c6e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 05 17:51:27.241220 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53981827-81c9-4d79-ae9c-26037450c6e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 05 17:51:27.243073 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53981827-81c9-4d79-ae9c-26037450c6e1 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 05 17:51:27.253050 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1492/2990] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Sep 5 17:51:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 05 17:51:37.249029 dv-cinder-855964-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=67 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}